NIRCam Imaging Pipeline Notebook#
Authors: B. Hilbert, based on the NIRISS imaging notebook by R. Diaz
Last Updated: November 18, 2024
Pipeline Version: 1.15.1 (Build 11.0)
Purpose:
This notebook provides a framework for processing generic Near-Infrared
Camera (NIRCam) Imaging data through all three James Webb Space Telescope
(JWST) pipeline stages. Data is assumed to be located in a folder structure
following the paths set up below. It should not be necessary to edit
any cells other than in the Configuration section unless
modifying the standard pipeline processing options.
Data:
This example is set up to use an example dataset is from
Program ID
2739 (PI: Pontoppidan) which is a Cycle 1 Outreach program.
We focus on the data from Observation 001 Visit 002, in which M-16, or the
“Pillars of Creation” were observed.
Example input data to use will be downloaded automatically unless
disabled (i.e., to use local files instead).
JWST pipeline version and CRDS context:
This notebook was written for the calibration pipeline version given
above. It sets the CRDS context to the latest context in the JWST
Calibration Reference Data System (CRDS) associated with that
pipeline version. If you use different pipeline versions or
CRDS context, please read the relevant release notes
(here for pipeline,
here for CRDS) for possibly relevant
changes.
Updates:
This notebook is regularly updated as improvements are made to the
pipeline. Find the most up to date version of this notebook at:
https://github.com/spacetelescope/jwst-pipeline-notebooks/
Recent Changes:
Sept 5, 2024: original notebook created
Nov 11, 2024: Comment out line to set the context
Nov 18, 2024: Do not require both SW and LW user-provided data
November 22, 2024: Updates to workflow when skipping pipeline modules
Table of Contents#
Configuration
Package Imports
Demo Mode Setup (ignore if not using demo data)
Directory Setup
Detector1 Pipeline
Image2 Pipeline
Image3 Pipeline
Visualize the resampled images
Visualize Detected Sources
Notes
1. Configuration#
Set basic configuration for runing notebook.
Install dependencies and parameters#
To make sure that the pipeline version is compatabile with the steps
discussed below and the required dependencies and packages are installed,
you can create a fresh conda environment and install the provided
requirements.txt file:
conda create -n nircam_imaging_pipeline python=3.11
conda activate nircam_imaging_pipeline
pip install -r requirements.txt
Set the basic parameters to use with this notebook. These will affect what data is used, where data is located (if already in disk), and pipeline modules run in this data. The list of parameters are:
demo_mode
directories with data
pipeline modules
# Basic import necessary for configuration
import os
Note that
demo_mode must be set appropriately below.
Set demo_mode = True to run in demonstration mode. In this
mode this notebook will download example data from the Barbara A.
Mikulski Archive for Space Telescopes
(MAST) and process it through
the pipeline. This will all happen in a local directory unless modified in
Section 3 below.
Set demo_mode = False if you want to process your own data
that has already been downloaded and provide the location of the data.
# Set parameters for demo_mode, channel, band, data mode directories, and
# processing steps.
# -----------------------------Demo Mode---------------------------------
demo_mode = True
if demo_mode:
print('Running in demonstration mode using online example data!')
# --------------------------User Mode Directories------------------------
# If demo_mode = False, look for user data in these paths
if not demo_mode:
# Set directory paths for processing specific data; these will need
# to be changed to your local directory setup (below are given as
# examples)
user_home_dir = os.path.expanduser('~')
# Point to where science observation data are
# Assumes uncalibrated data in <sci_dir>/uncal/ and results in stage1,
# stage2, stage3 directories
sci_dir = os.path.join(user_home_dir, 'PID2739/Obs001/')
# --------------------------Set Processing Steps--------------------------
# Individual pipeline stages can be turned on/off here. Note that a later
# stage won't be able to run unless data products have already been
# produced from the prior stage.
# Science processing
dodet1 = True # calwebb_detector1
doimage2 = True # calwebb_image2
doimage3 = True # calwebb_image3
Running in demonstration mode using online example data!
Set CRDS context and server#
Before importing CRDS and JWST modules, we need
to configure our environment. This includes defining a CRDS cache
directory in which to keep the reference files that will be used by the
calibration pipeline.
If the root directory for the local CRDS cache directory has not been set already, it will be set to create one in the home directory.
# ------------------------Set CRDS context and paths----------------------
# Each version of the calibration pipeline is associated with a specific CRDS
# context file. The pipeline will select the appropriate context file behind
# the scenes while running. However, if you wish to override the default context
# file and run the pipeline with a different context, you can set that using
# the CRDS_CONTEXT environment variable. Here we show how this is done,
# although we leave the line commented out in order to use the default context.
# If you wish to specify a different context, uncomment the line below.
#%env CRDS_CONTEXT jwst_1293.pmap
# Check whether the local CRDS cache directory has been set.
# If not, set it to the user home directory
if (os.getenv('CRDS_PATH') is None):
os.environ['CRDS_PATH'] = os.path.join(os.path.expanduser('~'), 'crds')
# Check whether the CRDS server URL has been set. If not, set it.
if (os.getenv('CRDS_SERVER_URL') is None):
os.environ['CRDS_SERVER_URL'] = 'https://jwst-crds.stsci.edu'
# Echo CRDS path in use
print(f"CRDS local filepath: {os.environ['CRDS_PATH']}")
print(f"CRDS file server: {os.environ['CRDS_SERVER_URL']}")
if os.getenv('CRDS_CONTEXT'):
print(f"CRDS CONTEXT: {os.environ['CRDS_CONTEXT']}")
CRDS local filepath: /home/runner/crds
CRDS file server: https://jwst-crds.stsci.edu
2. Package Imports#
# Use the entire available screen width for this notebook
from IPython.display import display, HTML
display(HTML("<style>.container { width:95% !important; }</style>"))
# Basic system utilities for interacting with files
# ----------------------General Imports------------------------------------
import glob
import time
from pathlib import Path
# Numpy for doing calculations
import numpy as np
# To display full ouptut of cell, not just the last result
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
# -----------------------Astroquery Imports--------------------------------
# ASCII files, and downloading demo files
from astroquery.mast import Observations
# Astropy routines for visualizing detected sources:
from astropy.table import Table
from astropy.coordinates import SkyCoord
# ------------ Pipeline and Visualization Imports -----------------------
# for JWST calibration pipeline
import jwst
import crds
from jwst.pipeline import Detector1Pipeline
from jwst.pipeline import Image2Pipeline
from jwst.pipeline import Image3Pipeline
# JWST pipeline utilities
from asdf import AsdfFile
from jwst import datamodels
from jwst.associations import asn_from_list # Tools for creating association files
from jwst.associations.lib.rules_level3_base import DMS_Level3_Base # Definition of a Lvl3 association file
# For visualizing images
from jdaviz import Imviz
# Echo pipeline version and CRDS context in use
print(f"JWST Calibration Pipeline Version: {jwst.__version__}")
print(f"Using CRDS Context: {crds.get_context_name('jwst')}")
JWST Calibration Pipeline Version: 1.15.1
CRDS - INFO - Calibration SW Found: jwst 1.15.1 (/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst-1.15.1.dist-info)
Using CRDS Context: jwst_1293.pmap
# Start a timer to keep track of runtime
time0 = time.perf_counter()
3. Demo Mode Setup (ignore if not using demo data)#
If running in demonstration mode, set up the program information to
retrieve the uncalibrated data automatically from MAST using
astroquery.
MAST allows for flexibility of searching by the proposal ID and the
observation ID instead of just filenames.
For illustrative purposes, we focus on data taken using the NIRCam
F200W and F444W filters
and start with uncalibrated data products. The files are named
jw02739001002_02105_0000<dither>_nrc<det>_uncal.fits, where dither refers to the
dither step number, and det is the detector name. Through this notebook we will refer to data
with filter F200W as SW data and F444W as LW data.
More information about the JWST file naming conventions can be found at: https://jwst-pipeline.readthedocs.io/en/latest/jwst/data_products/file_naming.html
# Set up the program information and paths for demo program
if demo_mode:
print('Running in demonstration mode and will download example data from MAST!')
program = "02739"
sci_observtn = "001"
data_dir = os.path.join('.', 'nrc_im_demo_data')
download_dir = data_dir
sci_dir = os.path.join(data_dir, 'Obs' + sci_observtn)
uncal_dir = os.path.join(sci_dir, 'uncal')
# Ensure filepaths for input data exist
if not os.path.exists(uncal_dir):
os.makedirs(uncal_dir)
# Create directory if it does not exist
if not os.path.isdir(data_dir):
os.mkdir(data_dir)
Running in demonstration mode and will download example data from MAST!
Identify list of science (SCI) uncalibrated files associated with visits.
Work one filter at a time, so that we can more easily filter by detector and keep only the module A files.
First download the F200W data.
# Obtain a list of observation IDs for the specified demo program
if demo_mode:
# Science data
sci_obs_id_table = Observations.query_criteria(instrument_name=["NIRCAM/IMAGE"],
provenance_name=["CALJWST"], # Executed observations
filters=['F200W'], # Data for Specific Filter
obs_id=['jw' + program + '-o' + sci_observtn + '*']
)
if demo_mode:
sci_obs_id_table
Table masked=True length=1
| intentType | obs_collection | provenance_name | instrument_name | project | filters | wavelength_region | target_name | target_classification | obs_id | s_ra | s_dec | dataproduct_type | proposal_pi | calib_level | t_min | t_max | t_exptime | em_min | em_max | obs_title | t_obs_release | proposal_id | proposal_type | sequence_number | s_region | jpegURL | dataURL | dataRights | mtFlag | srcDen | obsid | objID |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| str7 | str4 | str7 | str12 | str4 | str5 | str8 | str4 | str42 | str36 | float64 | float64 | str5 | str21 | int64 | float64 | float64 | float64 | float64 | float64 | str30 | float64 | str4 | str2 | int64 | str117 | str62 | str63 | str6 | bool | float64 | str9 | str9 |
| science | JWST | CALJWST | NIRCAM/IMAGE | JWST | F200W | INFRARED | M-16 | ISM; Molecular clouds; Nebulae; Protostars | jw02739-o001_t001_nircam_clear-f200w | 274.7299074457347 | -13.851719782834468 | image | Pontoppidan, Klaus M. | 3 | 59805.40325079491 | 59824.93756190972 | 3221.0400000000004 | 1755.0 | 2226.0 | JWST Cycle 1 Outreach Campaign | 59871.55982626 | 2739 | DD | -- | POLYGON 274.693610607 -13.78846435 274.770282437 -13.790845449 274.766206195 -13.91520072 274.689493363 -13.912818346 | mast:JWST/product/jw02739-o001_t001_nircam_clear-f200w_i2d.jpg | mast:JWST/product/jw02739-o001_t001_nircam_clear-f200w_i2d.fits | PUBLIC | False | nan | 219392596 | 617131674 |
# Turn the list of visits into a list of uncalibrated data files
if demo_mode:
# Define types of files to select
file_dict = {'uncal': {'product_type': 'SCIENCE',
'productSubGroupDescription': 'UNCAL',
'calib_level': [1]}}
# Science files
sci_files_to_download = []
# Loop over visits identifying uncalibrated files that are associated
# with them
for exposure in (sci_obs_id_table):
products = Observations.get_product_list(exposure)
for filetype, query_dict in file_dict.items():
filtered_products = Observations.filter_products(products, productType=query_dict['product_type'],
productSubGroupDescription=query_dict['productSubGroupDescription'],
calib_level=query_dict['calib_level'])
sci_files_to_download.extend(filtered_products['dataURI'])
# To limit data volume, keep only files from visit 002, dithers 1 and 2, and only A-module
sw_sci_files_to_download = [fname for fname in sci_files_to_download if 'jw02739001002_02105' in fname and
('nrca2' in fname or 'nrca4' in fname) and ('00001' in fname or '00002' in fname)]
sw_sci_files_to_download = sorted(sw_sci_files_to_download)
print(f"Science files selected for downloading: {len(sw_sci_files_to_download)}")
Science files selected for downloading: 4
# List the SW files to download
if demo_mode:
sw_sci_files_to_download
['mast:JWST/product/jw02739001002_02105_00001_nrca2_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00001_nrca4_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00002_nrca2_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00002_nrca4_uncal.fits']
Now repeat the process for the F444W data.
# Obtain a list of observation IDs for the specified demo program
if demo_mode:
# Science data
sci_obs_id_table = Observations.query_criteria(instrument_name=["NIRCAM/IMAGE"],
provenance_name=["CALJWST"], # Executed observations
filters=['F444W'], # Data for Specific Filter
obs_id=['jw' + program + '-o' + sci_observtn + '*']
)
if demo_mode:
sci_obs_id_table
Table masked=True length=2
| intentType | obs_collection | provenance_name | instrument_name | project | filters | wavelength_region | target_name | target_classification | obs_id | s_ra | s_dec | dataproduct_type | proposal_pi | calib_level | t_min | t_max | t_exptime | em_min | em_max | obs_title | t_obs_release | proposal_id | proposal_type | sequence_number | s_region | jpegURL | dataURL | dataRights | mtFlag | srcDen | obsid | objID |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| str7 | str4 | str7 | str12 | str4 | str11 | str8 | str4 | str42 | str36 | float64 | float64 | str5 | str21 | int64 | float64 | float64 | float64 | float64 | float64 | str30 | float64 | str4 | str2 | int64 | str119 | str62 | str63 | str6 | bool | float64 | str8 | str9 |
| science | JWST | CALJWST | NIRCAM/IMAGE | JWST | F444W;F470N | INFRARED | M-16 | ISM; Molecular clouds; Nebulae; Protostars | jw02739-o001_t001_nircam_f444w-f470n | 274.7299074457283 | -13.851719782779508 | image | Pontoppidan, Klaus M. | 3 | 59805.35664999352 | 59824.89791972222 | 5797.86 | 3880.0 | 4986.0 | JWST Cycle 1 Outreach Campaign | 59871.55982626 | 2739 | DD | -- | POLYGON 274.694911886 -13.788673111 274.770074195 -13.791626152 274.764955179 -13.914928446 274.689753041 -13.911973838 | mast:JWST/product/jw02739-o001_t001_nircam_f444w-f470n_i2d.jpg | mast:JWST/product/jw02739-o001_t001_nircam_f444w-f470n_i2d.fits | PUBLIC | False | nan | 89121540 | 617131668 |
| science | JWST | CALJWST | NIRCAM/IMAGE | JWST | F444W | INFRARED | M-16 | ISM; Molecular clouds; Nebulae; Protostars | jw02739-o001_t001_nircam_clear-f444w | 274.7299074457347 | -13.851719782834468 | image | Pontoppidan, Klaus M. | 3 | 59805.403251535645 | 59824.93756190972 | 3221.0400000000004 | 3880.0 | 4986.0 | JWST Cycle 1 Outreach Campaign | 59871.55982626 | 2739 | DD | -- | POLYGON 274.694954069 -13.788646948 274.770047607 -13.791598372 274.764924567 -13.914951606 274.68979122 -13.911998615 | mast:JWST/product/jw02739-o001_t001_nircam_clear-f444w_i2d.jpg | mast:JWST/product/jw02739-o001_t001_nircam_clear-f444w_i2d.fits | PUBLIC | False | nan | 89121644 | 617131685 |
# Turn the list of visits into a list of uncalibrated data files
if demo_mode:
# Define types of files to select
file_dict = {'uncal': {'product_type': 'SCIENCE',
'productSubGroupDescription': 'UNCAL',
'calib_level': [1]}}
# Science files
sci_files_to_download = []
# Loop over visits identifying uncalibrated files that are associated
# with them
for exposure in (sci_obs_id_table):
products = Observations.get_product_list(exposure)
for filetype, query_dict in file_dict.items():
filtered_products = Observations.filter_products(products, productType=query_dict['product_type'],
productSubGroupDescription=query_dict['productSubGroupDescription'],
calib_level=query_dict['calib_level'])
sci_files_to_download.extend(filtered_products['dataURI'])
# To limit data volume, keep only files from visit 002, dithers 1 and 2, and only A-module
lw_sci_files_to_download = [fname for fname in sci_files_to_download if 'jw02739001002_02105' in fname and
'nrca' in fname and ('00001' in fname or '00002' in fname)]
lw_sci_files_to_download = sorted(lw_sci_files_to_download)
print(f"Science files selected for downloading: {len(lw_sci_files_to_download)}")
Science files selected for downloading: 2
# List the LW files to download
if demo_mode:
lw_sci_files_to_download
['mast:JWST/product/jw02739001002_02105_00001_nrcalong_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00002_nrcalong_uncal.fits']
# Full list the science files to download
if demo_mode:
sci_files_to_download = sw_sci_files_to_download + lw_sci_files_to_download
sci_files_to_download
['mast:JWST/product/jw02739001002_02105_00001_nrca2_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00001_nrca4_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00002_nrca2_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00002_nrca4_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00001_nrcalong_uncal.fits',
'mast:JWST/product/jw02739001002_02105_00002_nrcalong_uncal.fits']
Download all the uncal files and place them into the appropriate directories.
Warning: If this notebook is halted during this step the downloaded file
may be incomplete, and cause crashes later on!
# Download the demo data if it does not already exist
if demo_mode:
for filename in sci_files_to_download:
sci_manifest = Observations.download_file(filename,
local_path=os.path.join(uncal_dir, Path(filename).name))
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00001_nrca2_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrca2_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00001_nrca4_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrca4_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00002_nrca2_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrca2_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00002_nrca4_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrca4_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00001_nrcalong_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrcalong_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00002_nrcalong_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrcalong_uncal.fits ...
[Done]
4. Directory Setup#
Set up detailed paths to input/output stages here.
# Define output subdirectories to keep science data products organized
uncal_dir = os.path.join(sci_dir, 'uncal') # Uncalibrated pipeline inputs should be here
det1_dir = os.path.join(sci_dir, 'stage1') # calwebb_detector1 pipeline outputs will go here
image2_dir = os.path.join(sci_dir, 'stage2') # calwebb_spec2 pipeline outputs will go here
image3_dir = os.path.join(sci_dir, 'stage3') # calwebb_spec3 pipeline outputs will go here
# We need to check that the desired output directories exist, and if not
# create them
if not os.path.exists(det1_dir):
os.makedirs(det1_dir)
if not os.path.exists(image2_dir):
os.makedirs(image2_dir)
if not os.path.exists(image3_dir):
os.makedirs(image3_dir)
Look at the first file to determine exposure parameters and practice using JWST datamodels¶
# List uncal files
uncal_files = sorted(glob.glob(os.path.join(uncal_dir, '*_uncal.fits')))
# Separate SW from LW files
sw_uncal_files = [uncfile for uncfile in uncal_files if 'long' not in uncfile]
lw_uncal_files = [uncfile for uncfile in uncal_files if 'long' in uncfile]
colnames = ('Instrument', 'Filter', 'Pupil', 'Number of Integrations', 'Number of Groups',
'Readout pattern', 'Dither position number')
dtypes = ('S7', 'S10', 'S10', 'i4', 'i4', 'S15', 'i4')
meta_check = Table(names=(colnames), dtype=dtypes)
# Open example files and get metadata for display
if len(sw_uncal_files) > 0:
sw_examine = datamodels.open(sw_uncal_files[0])
sw_row = [sw_examine.meta.instrument.name, sw_examine.meta.instrument.filter,
sw_examine.meta.instrument.pupil, sw_examine.meta.exposure.nints,
sw_examine.meta.exposure.ngroups, sw_examine.meta.exposure.readpatt,
sw_examine.meta.dither.position_number]
meta_check.add_row(sw_row)
if len(lw_uncal_files) > 0:
lw_examine = datamodels.open(lw_uncal_files[0])
lw_row = [lw_examine.meta.instrument.name, lw_examine.meta.instrument.filter,
lw_examine.meta.instrument.pupil, lw_examine.meta.exposure.nints,
lw_examine.meta.exposure.ngroups, lw_examine.meta.exposure.readpatt,
lw_examine.meta.dither.position_number]
meta_check.add_row(lw_row)
# Print out exposure info
meta_check
Table length=2
| Instrument | Filter | Pupil | Number of Integrations | Number of Groups | Readout pattern | Dither position number |
|---|---|---|---|---|---|---|
| bytes7 | bytes10 | bytes10 | int32 | int32 | bytes15 | int32 |
| NIRCAM | F200W | CLEAR | 1 | 8 | BRIGHT1 | 1 |
| NIRCAM | F444W | CLEAR | 1 | 8 | BRIGHT1 | 1 |
The table above shows basic exposure information from the first shortwave as well as the first longwave file. When using
the demo data, we confirm that the data file is for the NIRCam instrument
using the F200W and F444W filters in the Filter Wheel
crossed with the CLEAR filter in the Pupil Wheel. This observation uses
the BRIGHT1 readout pattern,
8 groups per integration, and 1 integration per exposure. This data file
is the 1st dither position in this exposure sequence. For more information
about how JWST exposures are defined by up-the-ramp sampling, see the
Understanding Exposure Times JDox article.
This metadata will be the same for all exposures in this observation, except for the dither position number.
# Print out the time benchmark
time_det1 = time.perf_counter()
print(f"Runtime so far: {time_det1 - time0:0.0f} seconds")
Runtime so far: 29 seconds
5. Detector1 Pipeline#
Run the datasets through the
Detector1
stage of the pipeline to apply detector level calibrations and create a
countrate data product where slopes are fitted to the integration ramps.
These *_rate.fits products are 2D (nrows x ncols), averaged over all
integrations. 3D countrate data products (*_rateints.fits) are also
created (nintegrations x nrows x ncols) which have the fitted ramp slopes
for each integration.
By default, all steps in the Detector1 stage of the pipeline are run for
NIRCam except the ipc correction step and the gain_scale step. Note
that the persistence step
has been turned off by default starting with CRDS context jwst_1264.pmap.
This step does not automatically correct the science data for persistence.
The persistence step creates a *_trapsfilled.fits file which is a model
that records the number of traps filled at each pixel at the end of an exposure.
This file would be used as an input to the persistence step, via the input_trapsfilled
argument, to correct the subsequent science exposure for persistence. Since persistence
is not well calibrated for NIRCam, the step has been turned off in order to speed up
calibration and to not create empty *_trapsfilled.fits files. This step
can be turned on when running the pipeline in Python by doing:
rate_result = Detector1Pipeline.call(uncal, steps={'persistence': {'skip': False}})
or as indicated in the cell bellow using a dictionary.
As of CRDS context jwst_1155.pmap and later, the
jump step
of the Detector1 stage of the pipeline will remove signal associated
with snowballs
in the NIRCam imaging mode. This correction is turned on using the parameter
expand_large_events=True. This and other parameters related to the snowball correction
are specified in the pars-jumpstep parameter reference file. Users may wish to alter
parameters to optimize removal of snowball residuals. Available parameters are discussed
in the Detection and Flagging of Showers and Snowballs in JWST Technical Report (Regan 2023).
# Set up a dictionary to define how the Detector1 pipeline should be configured
# Boilerplate dictionary setup
det1dict = {}
det1dict['group_scale'], det1dict['dq_init'], det1dict['saturation'] = {}, {}, {}
det1dict['ipc'], det1dict['superbias'], det1dict['refpix'] = {}, {}, {}
det1dict['linearity'], det1dict['persistence'], det1dict['dark_current'], = {}, {}, {}
det1dict['charge_migration'], det1dict['jump'], det1dict['ramp_fit'] = {}, {}, {}
det1dict['gain_scale'] = {}
# Overrides for whether or not certain steps should be skipped
# skipping the persistence step
det1dict['persistence']['skip'] = True
# Overrides for various reference files
# Files should be in the base local directory or provide full path
#det1dict['dq_init']['override_mask'] = 'myfile.fits' # Bad pixel mask
#det1dict['saturation']['override_saturation'] = 'myfile.fits' # Saturation
#det1dict['reset']['override_reset'] = 'myfile.fits' # Reset
#det1dict['linearity']['override_linearity'] = 'myfile.fits' # Linearity
#det1dict['rscd']['override_rscd'] = 'myfile.fits' # RSCD
#det1dict['dark_current']['override_dark'] = 'myfile.fits' # Dark current subtraction
#det1dict['jump']['override_gain'] = 'myfile.fits' # Gain used by jump step
#det1dict['ramp_fit']['override_gain'] = 'myfile.fits' # Gain used by ramp fitting step
#det1dict['jump']['override_readnoise'] = 'myfile.fits' # Read noise used by jump step
#det1dict['ramp_fit']['override_readnoise'] = 'myfile.fits' # Read noise used by ramp fitting step
# Turn on multi-core processing (This is off by default). Choose what fraction
# of cores to use (quarter, half, all, or an integer number)
det1dict['jump']['maximum_cores'] = 'half'
# Explicitly turn on snowball correction. (Even though it is on by default)
det1dict['jump']['expand_large_events'] = True
Run the Detector1 pipeline on all input data, regardless of filter.
# Run Detector1 stage of pipeline, specifying:
# output directory to save *_rate.fits files
# save_results flag set to True so the rate files are saved
if dodet1:
for uncal in uncal_files:
rate_result = Detector1Pipeline.call(uncal, output_dir=det1_dir, steps=det1dict, save_results=True)
else:
print('Skipping Detector1 processing')
2024-12-09 18:13:54,159 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_system_datalvl_0002.rmap 694 bytes (1 / 199 files) (0 / 716.1 K bytes)
2024-12-09 18:13:54,499 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_system_calver_0038.rmap 4.8 K bytes (2 / 199 files) (694 / 716.1 K bytes)
2024-12-09 18:13:54,746 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_system_0037.imap 385 bytes (3 / 199 files) (5.5 K / 716.1 K bytes)
2024-12-09 18:13:54,988 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_wavelengthrange_0024.rmap 1.4 K bytes (4 / 199 files) (5.9 K / 716.1 K bytes)
2024-12-09 18:13:55,227 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_wavecorr_0005.rmap 884 bytes (5 / 199 files) (7.3 K / 716.1 K bytes)
2024-12-09 18:13:55,487 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_superbias_0074.rmap 33.8 K bytes (6 / 199 files) (8.2 K / 716.1 K bytes)
2024-12-09 18:13:55,799 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_sflat_0026.rmap 20.6 K bytes (7 / 199 files) (41.9 K / 716.1 K bytes)
2024-12-09 18:13:56,121 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_saturation_0018.rmap 2.0 K bytes (8 / 199 files) (62.5 K / 716.1 K bytes)
2024-12-09 18:13:56,358 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_refpix_0015.rmap 1.6 K bytes (9 / 199 files) (64.5 K / 716.1 K bytes)
2024-12-09 18:13:56,626 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_readnoise_0025.rmap 2.6 K bytes (10 / 199 files) (66.1 K / 716.1 K bytes)
2024-12-09 18:13:56,884 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_photom_0013.rmap 958 bytes (11 / 199 files) (68.7 K / 716.1 K bytes)
2024-12-09 18:13:57,132 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pathloss_0007.rmap 1.1 K bytes (12 / 199 files) (69.6 K / 716.1 K bytes)
2024-12-09 18:13:57,372 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-whitelightstep_0001.rmap 777 bytes (13 / 199 files) (70.8 K / 716.1 K bytes)
2024-12-09 18:13:57,609 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-spec2pipeline_0013.rmap 2.1 K bytes (14 / 199 files) (71.5 K / 716.1 K bytes)
2024-12-09 18:13:57,850 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-resamplespecstep_0002.rmap 709 bytes (15 / 199 files) (73.6 K / 716.1 K bytes)
2024-12-09 18:13:58,087 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-outlierdetectionstep_0002.rmap 852 bytes (16 / 199 files) (74.4 K / 716.1 K bytes)
2024-12-09 18:13:58,324 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-jumpstep_0005.rmap 810 bytes (17 / 199 files) (75.2 K / 716.1 K bytes)
2024-12-09 18:13:58,564 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-image2pipeline_0008.rmap 1.0 K bytes (18 / 199 files) (76.0 K / 716.1 K bytes)
2024-12-09 18:13:58,803 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-detector1pipeline_0003.rmap 1.1 K bytes (19 / 199 files) (77.0 K / 716.1 K bytes)
2024-12-09 18:13:59,042 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-darkpipeline_0003.rmap 872 bytes (20 / 199 files) (78.1 K / 716.1 K bytes)
2024-12-09 18:13:59,293 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-darkcurrentstep_0001.rmap 622 bytes (21 / 199 files) (78.9 K / 716.1 K bytes)
2024-12-09 18:13:59,532 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ote_0030.rmap 1.3 K bytes (22 / 199 files) (79.6 K / 716.1 K bytes)
2024-12-09 18:13:59,770 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_msaoper_0014.rmap 1.4 K bytes (23 / 199 files) (80.8 K / 716.1 K bytes)
2024-12-09 18:14:00,009 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_msa_0027.rmap 1.3 K bytes (24 / 199 files) (82.3 K / 716.1 K bytes)
2024-12-09 18:14:00,248 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_mask_0039.rmap 2.7 K bytes (25 / 199 files) (83.5 K / 716.1 K bytes)
2024-12-09 18:14:00,486 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_linearity_0017.rmap 1.6 K bytes (26 / 199 files) (86.2 K / 716.1 K bytes)
2024-12-09 18:14:00,726 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ipc_0006.rmap 876 bytes (27 / 199 files) (87.8 K / 716.1 K bytes)
2024-12-09 18:14:00,966 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ifuslicer_0017.rmap 1.5 K bytes (28 / 199 files) (88.6 K / 716.1 K bytes)
2024-12-09 18:14:01,203 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ifupost_0019.rmap 1.5 K bytes (29 / 199 files) (90.2 K / 716.1 K bytes)
2024-12-09 18:14:01,442 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ifufore_0017.rmap 1.5 K bytes (30 / 199 files) (91.7 K / 716.1 K bytes)
2024-12-09 18:14:01,680 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_gain_0023.rmap 1.8 K bytes (31 / 199 files) (93.2 K / 716.1 K bytes)
2024-12-09 18:14:01,920 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_fpa_0028.rmap 1.3 K bytes (32 / 199 files) (94.9 K / 716.1 K bytes)
2024-12-09 18:14:02,162 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_fore_0026.rmap 5.0 K bytes (33 / 199 files) (96.2 K / 716.1 K bytes)
2024-12-09 18:14:02,399 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_flat_0015.rmap 3.8 K bytes (34 / 199 files) (101.1 K / 716.1 K bytes)
2024-12-09 18:14:02,637 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_fflat_0024.rmap 7.2 K bytes (35 / 199 files) (105.0 K / 716.1 K bytes)
2024-12-09 18:14:02,876 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_extract1d_0018.rmap 2.3 K bytes (36 / 199 files) (112.2 K / 716.1 K bytes)
2024-12-09 18:14:03,115 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_disperser_0028.rmap 5.7 K bytes (37 / 199 files) (114.5 K / 716.1 K bytes)
2024-12-09 18:14:03,353 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_dflat_0007.rmap 1.1 K bytes (38 / 199 files) (120.2 K / 716.1 K bytes)
2024-12-09 18:14:03,595 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_dark_0069.rmap 32.6 K bytes (39 / 199 files) (121.3 K / 716.1 K bytes)
2024-12-09 18:14:03,909 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_cubepar_0015.rmap 966 bytes (40 / 199 files) (153.9 K / 716.1 K bytes)
2024-12-09 18:14:04,147 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_collimator_0026.rmap 1.3 K bytes (41 / 199 files) (154.9 K / 716.1 K bytes)
2024-12-09 18:14:04,390 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_camera_0026.rmap 1.3 K bytes (42 / 199 files) (156.2 K / 716.1 K bytes)
2024-12-09 18:14:04,628 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_barshadow_0007.rmap 1.8 K bytes (43 / 199 files) (157.5 K / 716.1 K bytes)
2024-12-09 18:14:04,866 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_area_0018.rmap 6.3 K bytes (44 / 199 files) (159.3 K / 716.1 K bytes)
2024-12-09 18:14:05,106 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_apcorr_0009.rmap 5.6 K bytes (45 / 199 files) (165.5 K / 716.1 K bytes)
2024-12-09 18:14:05,345 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_0379.imap 5.6 K bytes (46 / 199 files) (171.1 K / 716.1 K bytes)
2024-12-09 18:14:05,584 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_wfssbkg_0007.rmap 3.1 K bytes (47 / 199 files) (176.7 K / 716.1 K bytes)
2024-12-09 18:14:05,823 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_wavemap_0008.rmap 2.2 K bytes (48 / 199 files) (179.8 K / 716.1 K bytes)
2024-12-09 18:14:06,061 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_wavelengthrange_0006.rmap 862 bytes (49 / 199 files) (182.0 K / 716.1 K bytes)
2024-12-09 18:14:06,298 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_trappars_0004.rmap 753 bytes (50 / 199 files) (182.9 K / 716.1 K bytes)
2024-12-09 18:14:06,536 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_trapdensity_0005.rmap 705 bytes (51 / 199 files) (183.7 K / 716.1 K bytes)
2024-12-09 18:14:06,774 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_throughput_0005.rmap 1.3 K bytes (52 / 199 files) (184.4 K / 716.1 K bytes)
2024-12-09 18:14:07,010 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_superbias_0028.rmap 6.5 K bytes (53 / 199 files) (185.6 K / 716.1 K bytes)
2024-12-09 18:14:07,269 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_specwcs_0014.rmap 3.1 K bytes (54 / 199 files) (192.1 K / 716.1 K bytes)
2024-12-09 18:14:07,508 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_spectrace_0008.rmap 2.3 K bytes (55 / 199 files) (195.2 K / 716.1 K bytes)
2024-12-09 18:14:07,746 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_specprofile_0008.rmap 2.4 K bytes (56 / 199 files) (197.6 K / 716.1 K bytes)
2024-12-09 18:14:07,985 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_speckernel_0006.rmap 1.0 K bytes (57 / 199 files) (199.9 K / 716.1 K bytes)
2024-12-09 18:14:08,223 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_saturation_0015.rmap 829 bytes (58 / 199 files) (201.0 K / 716.1 K bytes)
2024-12-09 18:14:08,464 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_readnoise_0011.rmap 987 bytes (59 / 199 files) (201.8 K / 716.1 K bytes)
2024-12-09 18:14:08,702 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_photom_0035.rmap 1.3 K bytes (60 / 199 files) (202.8 K / 716.1 K bytes)
2024-12-09 18:14:08,940 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_persat_0007.rmap 674 bytes (61 / 199 files) (204.0 K / 716.1 K bytes)
2024-12-09 18:14:09,177 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pathloss_0003.rmap 758 bytes (62 / 199 files) (204.7 K / 716.1 K bytes)
2024-12-09 18:14:09,419 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-undersamplecorrectionstep_0001.rmap 904 bytes (63 / 199 files) (205.5 K / 716.1 K bytes)
2024-12-09 18:14:09,659 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-tweakregstep_0012.rmap 3.1 K bytes (64 / 199 files) (206.4 K / 716.1 K bytes)
2024-12-09 18:14:09,897 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-spec2pipeline_0008.rmap 984 bytes (65 / 199 files) (209.5 K / 716.1 K bytes)
2024-12-09 18:14:10,135 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-sourcecatalogstep_0002.rmap 2.3 K bytes (66 / 199 files) (210.5 K / 716.1 K bytes)
2024-12-09 18:14:10,372 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-resamplestep_0002.rmap 687 bytes (67 / 199 files) (212.8 K / 716.1 K bytes)
2024-12-09 18:14:10,610 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-outlierdetectionstep_0004.rmap 2.7 K bytes (68 / 199 files) (213.5 K / 716.1 K bytes)
2024-12-09 18:14:10,858 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-jumpstep_0007.rmap 6.4 K bytes (69 / 199 files) (216.1 K / 716.1 K bytes)
2024-12-09 18:14:11,097 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-image2pipeline_0005.rmap 1.0 K bytes (70 / 199 files) (222.5 K / 716.1 K bytes)
2024-12-09 18:14:11,335 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-detector1pipeline_0002.rmap 1.0 K bytes (71 / 199 files) (223.5 K / 716.1 K bytes)
2024-12-09 18:14:11,582 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-darkpipeline_0002.rmap 868 bytes (72 / 199 files) (224.6 K / 716.1 K bytes)
2024-12-09 18:14:11,821 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-darkcurrentstep_0001.rmap 591 bytes (73 / 199 files) (225.4 K / 716.1 K bytes)
2024-12-09 18:14:12,071 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-chargemigrationstep_0004.rmap 5.7 K bytes (74 / 199 files) (226.0 K / 716.1 K bytes)
2024-12-09 18:14:12,312 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_nrm_0002.rmap 663 bytes (75 / 199 files) (231.7 K / 716.1 K bytes)
2024-12-09 18:14:12,553 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_mask_0020.rmap 859 bytes (76 / 199 files) (232.3 K / 716.1 K bytes)
2024-12-09 18:14:12,797 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_linearity_0022.rmap 961 bytes (77 / 199 files) (233.2 K / 716.1 K bytes)
2024-12-09 18:14:13,037 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_ipc_0007.rmap 651 bytes (78 / 199 files) (234.2 K / 716.1 K bytes)
2024-12-09 18:14:13,274 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_gain_0011.rmap 797 bytes (79 / 199 files) (234.8 K / 716.1 K bytes)
2024-12-09 18:14:13,512 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_flat_0023.rmap 5.9 K bytes (80 / 199 files) (235.6 K / 716.1 K bytes)
2024-12-09 18:14:13,748 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_filteroffset_0010.rmap 853 bytes (81 / 199 files) (241.5 K / 716.1 K bytes)
2024-12-09 18:14:13,987 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_extract1d_0007.rmap 905 bytes (82 / 199 files) (242.3 K / 716.1 K bytes)
2024-12-09 18:14:14,227 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_drizpars_0004.rmap 519 bytes (83 / 199 files) (243.2 K / 716.1 K bytes)
2024-12-09 18:14:14,463 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_distortion_0025.rmap 3.4 K bytes (84 / 199 files) (243.8 K / 716.1 K bytes)
2024-12-09 18:14:14,700 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_dark_0031.rmap 6.8 K bytes (85 / 199 files) (247.2 K / 716.1 K bytes)
2024-12-09 18:14:14,936 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_area_0014.rmap 2.7 K bytes (86 / 199 files) (254.0 K / 716.1 K bytes)
2024-12-09 18:14:15,173 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_apcorr_0010.rmap 4.3 K bytes (87 / 199 files) (256.7 K / 716.1 K bytes)
2024-12-09 18:14:15,411 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_abvegaoffset_0004.rmap 1.4 K bytes (88 / 199 files) (261.0 K / 716.1 K bytes)
2024-12-09 18:14:15,651 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_0256.imap 5.6 K bytes (89 / 199 files) (262.3 K / 716.1 K bytes)
2024-12-09 18:14:15,889 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_wfssbkg_0004.rmap 7.2 K bytes (90 / 199 files) (268.0 K / 716.1 K bytes)
2024-12-09 18:14:16,128 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_wavelengthrange_0010.rmap 996 bytes (91 / 199 files) (275.2 K / 716.1 K bytes)
2024-12-09 18:14:16,364 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_tsophot_0003.rmap 896 bytes (92 / 199 files) (276.2 K / 716.1 K bytes)
2024-12-09 18:14:16,601 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_trappars_0003.rmap 1.6 K bytes (93 / 199 files) (277.0 K / 716.1 K bytes)
2024-12-09 18:14:16,839 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_trapdensity_0003.rmap 1.6 K bytes (94 / 199 files) (278.6 K / 716.1 K bytes)
2024-12-09 18:14:17,076 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_superbias_0017.rmap 16.1 K bytes (95 / 199 files) (280.3 K / 716.1 K bytes)
2024-12-09 18:14:17,389 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_specwcs_0022.rmap 7.1 K bytes (96 / 199 files) (296.3 K / 716.1 K bytes)
2024-12-09 18:14:17,628 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_saturation_0010.rmap 2.2 K bytes (97 / 199 files) (303.4 K / 716.1 K bytes)
2024-12-09 18:14:17,866 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_readnoise_0024.rmap 23.1 K bytes (98 / 199 files) (305.6 K / 716.1 K bytes)
2024-12-09 18:14:18,179 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_psfmask_0008.rmap 28.4 K bytes (99 / 199 files) (328.7 K / 716.1 K bytes)
2024-12-09 18:14:18,491 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_photom_0028.rmap 3.4 K bytes (100 / 199 files) (357.0 K / 716.1 K bytes)
2024-12-09 18:14:18,729 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_persat_0005.rmap 1.6 K bytes (101 / 199 files) (360.4 K / 716.1 K bytes)
2024-12-09 18:14:18,968 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-whitelightstep_0003.rmap 1.5 K bytes (102 / 199 files) (361.9 K / 716.1 K bytes)
2024-12-09 18:14:19,205 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-tweakregstep_0003.rmap 4.5 K bytes (103 / 199 files) (363.4 K / 716.1 K bytes)
2024-12-09 18:14:19,442 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-spec2pipeline_0008.rmap 984 bytes (104 / 199 files) (367.9 K / 716.1 K bytes)
2024-12-09 18:14:19,680 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-sourcecatalogstep_0002.rmap 4.6 K bytes (105 / 199 files) (368.9 K / 716.1 K bytes)
2024-12-09 18:14:19,920 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-resamplestep_0002.rmap 687 bytes (106 / 199 files) (373.5 K / 716.1 K bytes)
2024-12-09 18:14:20,159 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-outlierdetectionstep_0003.rmap 940 bytes (107 / 199 files) (374.2 K / 716.1 K bytes)
2024-12-09 18:14:20,399 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-jumpstep_0005.rmap 806 bytes (108 / 199 files) (375.1 K / 716.1 K bytes)
2024-12-09 18:14:20,639 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-image2pipeline_0003.rmap 1.0 K bytes (109 / 199 files) (375.9 K / 716.1 K bytes)
2024-12-09 18:14:20,877 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-detector1pipeline_0003.rmap 1.0 K bytes (110 / 199 files) (377.0 K / 716.1 K bytes)
2024-12-09 18:14:21,115 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-darkpipeline_0002.rmap 868 bytes (111 / 199 files) (378.0 K / 716.1 K bytes)
2024-12-09 18:14:21,352 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-darkcurrentstep_0001.rmap 618 bytes (112 / 199 files) (378.9 K / 716.1 K bytes)
2024-12-09 18:14:21,592 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_mask_0011.rmap 3.5 K bytes (113 / 199 files) (379.5 K / 716.1 K bytes)
2024-12-09 18:14:21,831 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_linearity_0011.rmap 2.4 K bytes (114 / 199 files) (383.0 K / 716.1 K bytes)
2024-12-09 18:14:22,069 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_ipc_0003.rmap 2.0 K bytes (115 / 199 files) (385.4 K / 716.1 K bytes)
2024-12-09 18:14:22,311 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_gain_0016.rmap 2.1 K bytes (116 / 199 files) (387.4 K / 716.1 K bytes)
2024-12-09 18:14:22,548 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_flat_0027.rmap 51.7 K bytes (117 / 199 files) (389.5 K / 716.1 K bytes)
2024-12-09 18:14:22,930 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_filteroffset_0004.rmap 1.4 K bytes (118 / 199 files) (441.2 K / 716.1 K bytes)
2024-12-09 18:14:23,168 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_extract1d_0004.rmap 842 bytes (119 / 199 files) (442.6 K / 716.1 K bytes)
2024-12-09 18:14:23,405 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_drizpars_0001.rmap 519 bytes (120 / 199 files) (443.4 K / 716.1 K bytes)
2024-12-09 18:14:23,643 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_distortion_0033.rmap 53.4 K bytes (121 / 199 files) (444.0 K / 716.1 K bytes)
2024-12-09 18:14:24,027 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_dark_0044.rmap 26.3 K bytes (122 / 199 files) (497.3 K / 716.1 K bytes)
2024-12-09 18:14:24,342 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_area_0012.rmap 33.5 K bytes (123 / 199 files) (523.6 K / 716.1 K bytes)
2024-12-09 18:14:24,656 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_apcorr_0008.rmap 4.3 K bytes (124 / 199 files) (557.2 K / 716.1 K bytes)
2024-12-09 18:14:24,893 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_abvegaoffset_0003.rmap 1.3 K bytes (125 / 199 files) (561.4 K / 716.1 K bytes)
2024-12-09 18:14:25,131 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_0294.imap 5.4 K bytes (126 / 199 files) (562.7 K / 716.1 K bytes)
2024-12-09 18:14:25,373 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_wavelengthrange_0027.rmap 929 bytes (127 / 199 files) (568.1 K / 716.1 K bytes)
2024-12-09 18:14:25,610 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_tsophot_0004.rmap 882 bytes (128 / 199 files) (569.1 K / 716.1 K bytes)
2024-12-09 18:14:25,848 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_straymask_0009.rmap 987 bytes (129 / 199 files) (570.0 K / 716.1 K bytes)
2024-12-09 18:14:26,086 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_specwcs_0042.rmap 5.8 K bytes (130 / 199 files) (570.9 K / 716.1 K bytes)
2024-12-09 18:14:26,324 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_saturation_0015.rmap 1.2 K bytes (131 / 199 files) (576.7 K / 716.1 K bytes)
2024-12-09 18:14:26,560 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_rscd_0008.rmap 1.0 K bytes (132 / 199 files) (577.9 K / 716.1 K bytes)
2024-12-09 18:14:26,798 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_resol_0006.rmap 790 bytes (133 / 199 files) (578.9 K / 716.1 K bytes)
2024-12-09 18:14:27,034 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_reset_0026.rmap 3.9 K bytes (134 / 199 files) (579.7 K / 716.1 K bytes)
2024-12-09 18:14:27,271 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_regions_0033.rmap 5.2 K bytes (135 / 199 files) (583.6 K / 716.1 K bytes)
2024-12-09 18:14:27,514 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_readnoise_0023.rmap 1.6 K bytes (136 / 199 files) (588.8 K / 716.1 K bytes)
2024-12-09 18:14:27,752 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_psfmask_0009.rmap 2.1 K bytes (137 / 199 files) (590.4 K / 716.1 K bytes)
2024-12-09 18:14:27,995 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_photom_0056.rmap 3.7 K bytes (138 / 199 files) (592.6 K / 716.1 K bytes)
2024-12-09 18:14:28,234 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pathloss_0005.rmap 866 bytes (139 / 199 files) (596.3 K / 716.1 K bytes)
2024-12-09 18:14:28,472 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-whitelightstep_0003.rmap 912 bytes (140 / 199 files) (597.2 K / 716.1 K bytes)
2024-12-09 18:14:28,708 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-tweakregstep_0003.rmap 1.8 K bytes (141 / 199 files) (598.1 K / 716.1 K bytes)
2024-12-09 18:14:28,944 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-spec3pipeline_0008.rmap 816 bytes (142 / 199 files) (599.9 K / 716.1 K bytes)
2024-12-09 18:14:29,181 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-spec2pipeline_0012.rmap 1.3 K bytes (143 / 199 files) (600.7 K / 716.1 K bytes)
2024-12-09 18:14:29,419 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-sourcecatalogstep_0003.rmap 1.9 K bytes (144 / 199 files) (602.1 K / 716.1 K bytes)
2024-12-09 18:14:29,659 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-resamplestep_0002.rmap 677 bytes (145 / 199 files) (604.0 K / 716.1 K bytes)
2024-12-09 18:14:29,897 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-resamplespecstep_0002.rmap 706 bytes (146 / 199 files) (604.6 K / 716.1 K bytes)
2024-12-09 18:14:30,134 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-outlierdetectionstep_0017.rmap 3.4 K bytes (147 / 199 files) (605.3 K / 716.1 K bytes)
2024-12-09 18:14:30,372 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-jumpstep_0009.rmap 1.4 K bytes (148 / 199 files) (608.7 K / 716.1 K bytes)
2024-12-09 18:14:30,612 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-image2pipeline_0007.rmap 983 bytes (149 / 199 files) (610.2 K / 716.1 K bytes)
2024-12-09 18:14:30,850 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-extract1dstep_0002.rmap 728 bytes (150 / 199 files) (611.1 K / 716.1 K bytes)
2024-12-09 18:14:31,089 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-emicorrstep_0002.rmap 796 bytes (151 / 199 files) (611.9 K / 716.1 K bytes)
2024-12-09 18:14:31,326 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-detector1pipeline_0009.rmap 1.4 K bytes (152 / 199 files) (612.7 K / 716.1 K bytes)
2024-12-09 18:14:31,563 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-darkpipeline_0002.rmap 860 bytes (153 / 199 files) (614.1 K / 716.1 K bytes)
2024-12-09 18:14:31,800 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-darkcurrentstep_0002.rmap 683 bytes (154 / 199 files) (614.9 K / 716.1 K bytes)
2024-12-09 18:14:32,036 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_mrsxartcorr_0002.rmap 2.2 K bytes (155 / 199 files) (615.6 K / 716.1 K bytes)
2024-12-09 18:14:32,273 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_mrsptcorr_0005.rmap 2.0 K bytes (156 / 199 files) (617.8 K / 716.1 K bytes)
2024-12-09 18:14:32,515 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_mask_0023.rmap 3.5 K bytes (157 / 199 files) (619.7 K / 716.1 K bytes)
2024-12-09 18:14:32,752 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_linearity_0018.rmap 2.8 K bytes (158 / 199 files) (623.2 K / 716.1 K bytes)
2024-12-09 18:14:32,990 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_ipc_0008.rmap 700 bytes (159 / 199 files) (626.1 K / 716.1 K bytes)
2024-12-09 18:14:33,227 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_gain_0013.rmap 3.9 K bytes (160 / 199 files) (626.8 K / 716.1 K bytes)
2024-12-09 18:14:33,467 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_fringefreq_0003.rmap 1.4 K bytes (161 / 199 files) (630.7 K / 716.1 K bytes)
2024-12-09 18:14:33,707 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_fringe_0019.rmap 3.9 K bytes (162 / 199 files) (632.1 K / 716.1 K bytes)
2024-12-09 18:14:33,945 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_flat_0065.rmap 15.5 K bytes (163 / 199 files) (636.0 K / 716.1 K bytes)
2024-12-09 18:14:34,253 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_filteroffset_0025.rmap 2.5 K bytes (164 / 199 files) (651.5 K / 716.1 K bytes)
2024-12-09 18:14:34,512 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_extract1d_0020.rmap 1.4 K bytes (165 / 199 files) (654.0 K / 716.1 K bytes)
2024-12-09 18:14:34,750 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_emicorr_0003.rmap 663 bytes (166 / 199 files) (655.4 K / 716.1 K bytes)
2024-12-09 18:14:34,989 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_drizpars_0002.rmap 511 bytes (167 / 199 files) (656.0 K / 716.1 K bytes)
2024-12-09 18:14:35,225 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_distortion_0040.rmap 4.9 K bytes (168 / 199 files) (656.5 K / 716.1 K bytes)
2024-12-09 18:14:35,463 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_dark_0036.rmap 4.4 K bytes (169 / 199 files) (661.5 K / 716.1 K bytes)
2024-12-09 18:14:35,700 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_cubepar_0017.rmap 800 bytes (170 / 199 files) (665.8 K / 716.1 K bytes)
2024-12-09 18:14:35,938 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_area_0015.rmap 866 bytes (171 / 199 files) (666.6 K / 716.1 K bytes)
2024-12-09 18:14:36,174 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_apcorr_0019.rmap 5.0 K bytes (172 / 199 files) (667.5 K / 716.1 K bytes)
2024-12-09 18:14:36,413 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_abvegaoffset_0002.rmap 1.3 K bytes (173 / 199 files) (672.4 K / 716.1 K bytes)
2024-12-09 18:14:36,650 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_0415.imap 5.7 K bytes (174 / 199 files) (673.7 K / 716.1 K bytes)
2024-12-09 18:14:36,889 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_trappars_0004.rmap 903 bytes (175 / 199 files) (679.4 K / 716.1 K bytes)
2024-12-09 18:14:37,127 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_trapdensity_0006.rmap 930 bytes (176 / 199 files) (680.3 K / 716.1 K bytes)
2024-12-09 18:14:37,365 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_superbias_0017.rmap 3.8 K bytes (177 / 199 files) (681.2 K / 716.1 K bytes)
2024-12-09 18:14:37,605 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_saturation_0009.rmap 779 bytes (178 / 199 files) (685.0 K / 716.1 K bytes)
2024-12-09 18:14:37,841 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_readnoise_0011.rmap 1.3 K bytes (179 / 199 files) (685.8 K / 716.1 K bytes)
2024-12-09 18:14:38,078 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_photom_0014.rmap 1.1 K bytes (180 / 199 files) (687.0 K / 716.1 K bytes)
2024-12-09 18:14:38,315 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_persat_0006.rmap 884 bytes (181 / 199 files) (688.2 K / 716.1 K bytes)
2024-12-09 18:14:38,551 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-tweakregstep_0002.rmap 850 bytes (182 / 199 files) (689.1 K / 716.1 K bytes)
2024-12-09 18:14:38,788 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-sourcecatalogstep_0001.rmap 636 bytes (183 / 199 files) (689.9 K / 716.1 K bytes)
2024-12-09 18:14:39,025 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-outlierdetectionstep_0001.rmap 654 bytes (184 / 199 files) (690.5 K / 716.1 K bytes)
2024-12-09 18:14:39,261 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-image2pipeline_0005.rmap 974 bytes (185 / 199 files) (691.2 K / 716.1 K bytes)
2024-12-09 18:14:39,496 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-detector1pipeline_0002.rmap 1.0 K bytes (186 / 199 files) (692.2 K / 716.1 K bytes)
2024-12-09 18:14:39,731 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-darkpipeline_0002.rmap 856 bytes (187 / 199 files) (693.2 K / 716.1 K bytes)
2024-12-09 18:14:39,969 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_mask_0023.rmap 1.1 K bytes (188 / 199 files) (694.1 K / 716.1 K bytes)
2024-12-09 18:14:40,208 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_linearity_0015.rmap 925 bytes (189 / 199 files) (695.1 K / 716.1 K bytes)
2024-12-09 18:14:40,446 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_ipc_0003.rmap 614 bytes (190 / 199 files) (696.0 K / 716.1 K bytes)
2024-12-09 18:14:40,683 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_gain_0010.rmap 890 bytes (191 / 199 files) (696.6 K / 716.1 K bytes)
2024-12-09 18:14:40,920 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_flat_0009.rmap 1.1 K bytes (192 / 199 files) (697.5 K / 716.1 K bytes)
2024-12-09 18:14:41,157 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_distortion_0011.rmap 1.2 K bytes (193 / 199 files) (698.7 K / 716.1 K bytes)
2024-12-09 18:14:41,393 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_dark_0017.rmap 4.3 K bytes (194 / 199 files) (699.9 K / 716.1 K bytes)
2024-12-09 18:14:41,631 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_area_0010.rmap 1.2 K bytes (195 / 199 files) (704.2 K / 716.1 K bytes)
2024-12-09 18:14:41,867 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_apcorr_0004.rmap 4.0 K bytes (196 / 199 files) (705.3 K / 716.1 K bytes)
2024-12-09 18:14:42,104 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_abvegaoffset_0002.rmap 1.3 K bytes (197 / 199 files) (709.3 K / 716.1 K bytes)
2024-12-09 18:14:42,341 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_0116.imap 5.0 K bytes (198 / 199 files) (710.6 K / 716.1 K bytes)
2024-12-09 18:14:42,580 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_1293.pmap 580 bytes (199 / 199 files) (715.5 K / 716.1 K bytes)
2024-12-09 18:14:43,238 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2024-12-09 18:14:43,243 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf 1.8 K bytes (1 / 1 files) (0 / 1.8 K bytes)
2024-12-09 18:14:43,485 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 18:14:43,498 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf 1.7 K bytes (1 / 1 files) (0 / 1.7 K bytes)
2024-12-09 18:14:43,738 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 18:14:43,756 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 18:14:43,757 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 18:14:43,758 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 18:14:43,759 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 18:14:43,760 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 18:14:43,762 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 18:14:43,763 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 18:14:43,764 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 18:14:43,765 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 18:14:43,766 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 18:14:43,767 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 18:14:43,768 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 18:14:43,770 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 18:14:43,770 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 18:14:43,771 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 18:14:43,773 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 18:14:43,775 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 18:14:43,776 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 18:14:43,777 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 18:14:43,915 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrca2_uncal.fits',).
2024-12-09 18:14:43,936 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: False
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_intermediate_results: False
user_supplied_reffile: None
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
type: baseline
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 4.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: True
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
maximum_cores: '1'
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2024-12-09 18:14:44,017 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 18:14:44,021 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits 6.3 G bytes (1 / 7 files) (0 / 6.6 G bytes)
2024-12-09 18:18:50,304 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits 16.8 M bytes (2 / 7 files) (6.3 G / 6.6 G bytes)
2024-12-09 18:18:51,486 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits 151.0 M bytes (3 / 7 files) (6.3 G / 6.6 G bytes)
2024-12-09 18:18:57,240 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits 16.8 M bytes (4 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 18:18:58,421 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits 16.8 M bytes (5 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 18:19:00,140 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits 33.6 M bytes (6 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 18:19:01,775 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits 50.4 M bytes (7 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 18:19:03,831 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits'.
2024-12-09 18:19:03,832 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits'.
2024-12-09 18:19:03,832 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits'.
2024-12-09 18:19:03,833 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits'.
2024-12-09 18:19:03,833 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits'.
2024-12-09 18:19:03,834 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 18:19:03,834 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 18:19:03,835 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 18:19:03,835 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits'.
2024-12-09 18:19:03,835 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits'.
2024-12-09 18:19:03,837 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 18:19:04,151 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:04,212 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 18:19:04,213 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 18:19:04,214 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 18:19:04,357 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:04,375 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits
2024-12-09 18:19:04,583 - CRDS - INFO - Calibration SW Found: jwst 1.15.1 (/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst-1.15.1.dist-info)
2024-12-09 18:19:04,867 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 18:19:05,010 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:05,028 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits
2024-12-09 18:19:05,077 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:05,120 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:06,078 - stpipe.Detector1Pipeline.saturation - INFO - Detected 8870 saturated pixels
2024-12-09 18:19:06,096 - stpipe.Detector1Pipeline.saturation - INFO - Detected 1 A/D floor pixels
2024-12-09 18:19:06,102 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 18:19:06,246 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:06,247 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 18:19:06,379 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:06,397 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits
2024-12-09 18:19:06,574 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 18:19:06,717 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:06,774 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 18:19:06,775 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 18:19:06,775 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 18:19:06,775 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 18:19:06,776 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 18:19:06,776 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 18:19:06,777 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 18:19:06,777 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 18:19:08,326 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 18:19:08,469 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:08,488 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits
2024-12-09 18:19:08,554 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:08,554 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:08,555 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:08,556 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:08,556 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:08,557 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:08,557 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:08,981 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 18:19:09,127 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:09,128 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 18:19:09,259 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:09,278 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits
2024-12-09 18:19:10,603 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:10,641 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:19:10,661 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 18:19:10,662 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=187, nframes=1, groupgap=0
2024-12-09 18:19:11,238 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 18:19:11,388 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:11,388 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 18:19:11,524 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:11,532 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 18:19:11,533 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 18:19:11,543 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits
2024-12-09 18:19:11,567 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits
2024-12-09 18:19:11,759 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 18:19:11,789 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection
2024-12-09 18:19:14,378 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 18:19:17,651 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 63
2024-12-09 18:19:17,651 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 5.89158 sec
2024-12-09 18:19:17,703 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 6.170333
2024-12-09 18:19:17,705 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 18:19:17,853 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:17,881 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits
2024-12-09 18:19:17,881 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits
2024-12-09 18:19:17,909 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 18:19:17,909 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 18:19:18,086 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 18:19:20,724 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.6347413063049316
2024-12-09 18:19:20,940 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 18:19:21,083 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:21,111 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:19:21,111 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:19:21,113 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:19:21,254 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 18:19:21,283 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:19:21,284 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:19:21,285 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:19:21,440 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rateints.fits
2024-12-09 18:19:21,440 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 18:19:21,441 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:19:21,585 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rate.fits
2024-12-09 18:19:21,586 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 18:19:21,675 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2024-12-09 18:19:21,678 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 18:19:21,690 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 18:19:21,707 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 18:19:21,708 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 18:19:21,709 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 18:19:21,710 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 18:19:21,710 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 18:19:21,711 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 18:19:21,712 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 18:19:21,713 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 18:19:21,714 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 18:19:21,715 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 18:19:21,716 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 18:19:21,717 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 18:19:21,717 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 18:19:21,718 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 18:19:21,719 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 18:19:21,720 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 18:19:21,722 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 18:19:21,723 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 18:19:21,724 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 18:19:21,884 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrca4_uncal.fits',).
2024-12-09 18:19:21,904 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: False
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_intermediate_results: False
user_supplied_reffile: None
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
type: baseline
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 4.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: True
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
maximum_cores: '1'
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2024-12-09 18:19:21,979 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca4_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 18:19:21,983 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits 6.3 G bytes (1 / 7 files) (0 / 6.6 G bytes)
2024-12-09 18:22:47,291 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits 16.8 M bytes (2 / 7 files) (6.3 G / 6.6 G bytes)
2024-12-09 18:22:50,244 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits 151.0 M bytes (3 / 7 files) (6.3 G / 6.6 G bytes)
2024-12-09 18:23:22,708 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits 16.8 M bytes (4 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 18:23:29,669 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits 16.8 M bytes (5 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 18:23:33,267 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits 33.6 M bytes (6 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 18:23:37,717 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits 50.4 M bytes (7 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 18:23:41,794 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits'.
2024-12-09 18:23:41,795 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits'.
2024-12-09 18:23:41,795 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits'.
2024-12-09 18:23:41,796 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits'.
2024-12-09 18:23:41,796 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits'.
2024-12-09 18:23:41,797 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 18:23:41,797 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 18:23:41,798 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 18:23:41,798 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits'.
2024-12-09 18:23:41,798 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits'.
2024-12-09 18:23:41,800 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 18:23:42,101 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:42,158 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 18:23:42,158 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 18:23:42,160 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 18:23:42,303 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:42,321 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits
2024-12-09 18:23:42,507 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 18:23:42,652 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:42,671 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits
2024-12-09 18:23:42,713 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:42,756 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:43,646 - stpipe.Detector1Pipeline.saturation - INFO - Detected 5208 saturated pixels
2024-12-09 18:23:43,659 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2024-12-09 18:23:43,666 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 18:23:43,808 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:43,808 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 18:23:43,948 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:43,966 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits
2024-12-09 18:23:44,145 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 18:23:44,290 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:44,347 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 18:23:44,347 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 18:23:44,348 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 18:23:44,348 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 18:23:44,348 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 18:23:44,349 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 18:23:44,349 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 18:23:44,350 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 18:23:45,896 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 18:23:46,041 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:46,059 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits
2024-12-09 18:23:46,122 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:46,123 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:46,123 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:46,124 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:46,125 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:46,125 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:46,126 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:46,559 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 18:23:46,704 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:46,705 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 18:23:46,838 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:46,856 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits
2024-12-09 18:23:48,133 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:48,172 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:23:48,183 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 18:23:48,183 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=187, nframes=1, groupgap=0
2024-12-09 18:23:48,768 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 18:23:48,913 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:48,914 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 18:23:49,053 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:49,062 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 18:23:49,063 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 18:23:49,073 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits
2024-12-09 18:23:49,097 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits
2024-12-09 18:23:49,287 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 18:23:49,314 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection
2024-12-09 18:23:51,883 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 18:23:54,321 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 59
2024-12-09 18:23:54,322 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 5.0337 sec
2024-12-09 18:23:54,372 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 5.309444
2024-12-09 18:23:54,374 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 18:23:54,517 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:54,544 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits
2024-12-09 18:23:54,544 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits
2024-12-09 18:23:54,572 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 18:23:54,573 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 18:23:54,732 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 18:23:57,352 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.615572690963745
2024-12-09 18:23:57,562 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 18:23:57,716 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:57,744 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:23:57,745 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:23:57,747 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:23:57,904 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 18:23:57,933 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:23:57,934 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:23:57,935 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:23:58,088 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rateints.fits
2024-12-09 18:23:58,089 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 18:23:58,089 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:23:58,235 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rate.fits
2024-12-09 18:23:58,236 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 18:23:58,324 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2024-12-09 18:23:58,327 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 18:23:58,339 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 18:23:58,356 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 18:23:58,357 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 18:23:58,358 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 18:23:58,359 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 18:23:58,360 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 18:23:58,361 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 18:23:58,362 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 18:23:58,363 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 18:23:58,363 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 18:23:58,364 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 18:23:58,365 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 18:23:58,366 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 18:23:58,367 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 18:23:58,368 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 18:23:58,368 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 18:23:58,369 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 18:23:58,373 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 18:23:58,374 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 18:23:58,375 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 18:23:58,534 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrcalong_uncal.fits',).
2024-12-09 18:23:58,554 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: False
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_intermediate_results: False
user_supplied_reffile: None
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
type: baseline
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 4.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: True
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
maximum_cores: '1'
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2024-12-09 18:23:58,625 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrcalong_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 18:23:58,629 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits 4.4 G bytes (1 / 7 files) (0 / 4.7 G bytes)
2024-12-09 18:26:31,788 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits 16.8 M bytes (2 / 7 files) (4.4 G / 4.7 G bytes)
2024-12-09 18:26:33,040 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits 151.0 M bytes (3 / 7 files) (4.4 G / 4.7 G bytes)
2024-12-09 18:26:38,232 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits 16.8 M bytes (4 / 7 files) (4.5 G / 4.7 G bytes)
2024-12-09 18:26:39,569 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits 16.8 M bytes (5 / 7 files) (4.6 G / 4.7 G bytes)
2024-12-09 18:26:40,940 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits 33.6 M bytes (6 / 7 files) (4.6 G / 4.7 G bytes)
2024-12-09 18:26:42,701 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits 50.4 M bytes (7 / 7 files) (4.6 G / 4.7 G bytes)
2024-12-09 18:26:45,696 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits'.
2024-12-09 18:26:45,697 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2024-12-09 18:26:45,697 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits'.
2024-12-09 18:26:45,698 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits'.
2024-12-09 18:26:45,698 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits'.
2024-12-09 18:26:45,699 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 18:26:45,699 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 18:26:45,699 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 18:26:45,700 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits'.
2024-12-09 18:26:45,700 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits'.
2024-12-09 18:26:45,702 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 18:26:45,996 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:46,051 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 18:26:46,052 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 18:26:46,054 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 18:26:46,194 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:46,212 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits
2024-12-09 18:26:46,402 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 18:26:46,552 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:46,570 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits
2024-12-09 18:26:46,612 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:46,653 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:47,511 - stpipe.Detector1Pipeline.saturation - INFO - Detected 27972 saturated pixels
2024-12-09 18:26:47,525 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2024-12-09 18:26:47,531 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 18:26:47,676 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:47,676 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 18:26:47,812 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:47,831 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits
2024-12-09 18:26:48,011 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 18:26:48,156 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:48,214 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 18:26:48,214 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 18:26:48,215 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 18:26:48,215 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 18:26:48,215 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 18:26:48,216 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 18:26:48,216 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 18:26:48,216 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 18:26:49,704 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 18:26:49,849 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:49,867 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits
2024-12-09 18:26:49,940 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:49,941 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:49,941 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:49,942 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:49,942 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:49,943 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:49,944 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:26:50,419 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 18:26:50,565 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:50,566 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 18:26:50,699 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:50,718 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits
2024-12-09 18:26:52,357 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 18:26:52,358 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=130, nframes=1, groupgap=0
2024-12-09 18:26:52,811 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 18:26:52,957 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:52,957 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 18:26:53,094 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:26:53,103 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 18:26:53,103 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 18:26:53,114 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2024-12-09 18:26:53,137 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits
2024-12-09 18:26:53,328 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 18:26:53,355 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection
2024-12-09 18:26:56,979 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 18:27:08,474 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 246
2024-12-09 18:27:08,474 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 15.1456 sec
2024-12-09 18:27:08,525 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 15.421696
2024-12-09 18:27:08,527 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 18:27:08,676 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:27:08,703 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits
2024-12-09 18:27:08,704 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2024-12-09 18:27:08,732 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 18:27:08,732 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 18:27:08,996 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 18:27:11,628 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.6284172534942627
2024-12-09 18:27:11,837 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 18:27:11,985 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:27:12,013 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:27:12,013 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:27:12,015 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:27:12,159 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 18:27:12,188 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:27:12,188 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:27:12,190 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:27:12,341 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rateints.fits
2024-12-09 18:27:12,342 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 18:27:12,343 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:27:12,487 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rate.fits
2024-12-09 18:27:12,488 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 18:27:12,572 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2024-12-09 18:27:12,576 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 18:27:12,587 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 18:27:12,604 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 18:27:12,605 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 18:27:12,606 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 18:27:12,607 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 18:27:12,608 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 18:27:12,609 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 18:27:12,610 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 18:27:12,611 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 18:27:12,611 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 18:27:12,612 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 18:27:12,613 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 18:27:12,614 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 18:27:12,615 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 18:27:12,615 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 18:27:12,616 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 18:27:12,617 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 18:27:12,619 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 18:27:12,619 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 18:27:12,620 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 18:27:12,778 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrca2_uncal.fits',).
2024-12-09 18:27:12,799 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: False
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_intermediate_results: False
user_supplied_reffile: None
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
type: baseline
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 4.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: True
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
maximum_cores: '1'
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2024-12-09 18:27:12,872 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrca2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 18:27:12,876 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits'.
2024-12-09 18:27:12,876 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits'.
2024-12-09 18:27:12,876 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits'.
2024-12-09 18:27:12,877 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits'.
2024-12-09 18:27:12,877 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits'.
2024-12-09 18:27:12,878 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 18:27:12,878 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 18:27:12,879 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 18:27:12,879 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits'.
2024-12-09 18:27:12,879 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits'.
2024-12-09 18:27:12,881 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 18:27:13,161 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:13,216 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 18:27:13,217 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 18:27:13,218 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 18:27:13,361 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:13,379 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits
2024-12-09 18:27:13,556 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 18:27:13,708 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:13,726 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits
2024-12-09 18:27:13,766 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:13,806 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:14,645 - stpipe.Detector1Pipeline.saturation - INFO - Detected 9023 saturated pixels
2024-12-09 18:27:14,659 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2024-12-09 18:27:14,664 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 18:27:14,811 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:14,811 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 18:27:14,948 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:14,966 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits
2024-12-09 18:27:15,132 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 18:27:15,278 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:15,335 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 18:27:15,336 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 18:27:15,336 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 18:27:15,336 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 18:27:15,337 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 18:27:15,337 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 18:27:15,338 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 18:27:15,338 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 18:27:16,832 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 18:27:16,978 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:16,996 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits
2024-12-09 18:27:17,054 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:17,055 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:17,055 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:17,056 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:17,057 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:17,057 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:17,058 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:17,478 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 18:27:17,625 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:17,626 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 18:27:17,760 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:17,779 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits
2024-12-09 18:27:18,824 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:18,864 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:18,876 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 18:27:18,877 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=187, nframes=1, groupgap=0
2024-12-09 18:27:19,455 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 18:27:19,600 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:19,600 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 18:27:19,740 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:19,749 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 18:27:19,750 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 18:27:19,760 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits
2024-12-09 18:27:19,783 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits
2024-12-09 18:27:19,974 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 18:27:19,996 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection
2024-12-09 18:27:22,658 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 18:27:25,878 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 60
2024-12-09 18:27:25,879 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 5.90467 sec
2024-12-09 18:27:25,929 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 6.180209
2024-12-09 18:27:25,932 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 18:27:26,079 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:26,107 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits
2024-12-09 18:27:26,107 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits
2024-12-09 18:27:26,135 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 18:27:26,135 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 18:27:26,305 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 18:27:28,926 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.617466688156128
2024-12-09 18:27:29,136 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 18:27:29,283 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:29,311 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:27:29,312 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:27:29,313 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:27:29,458 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 18:27:29,488 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:27:29,489 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:27:29,491 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:27:29,643 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rateints.fits
2024-12-09 18:27:29,643 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 18:27:29,644 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:27:29,789 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rate.fits
2024-12-09 18:27:29,790 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 18:27:29,874 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2024-12-09 18:27:29,877 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 18:27:29,889 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 18:27:29,906 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 18:27:29,907 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 18:27:29,908 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 18:27:29,909 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 18:27:29,910 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 18:27:29,910 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 18:27:29,911 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 18:27:29,913 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 18:27:29,913 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 18:27:29,914 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 18:27:29,915 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 18:27:29,916 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 18:27:29,917 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 18:27:29,917 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 18:27:29,918 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 18:27:29,919 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 18:27:29,921 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 18:27:29,922 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 18:27:29,923 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 18:27:30,085 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrca4_uncal.fits',).
2024-12-09 18:27:30,105 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: False
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_intermediate_results: False
user_supplied_reffile: None
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
type: baseline
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 4.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: True
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
maximum_cores: '1'
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2024-12-09 18:27:30,177 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrca4_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 18:27:30,181 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits'.
2024-12-09 18:27:30,182 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits'.
2024-12-09 18:27:30,182 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits'.
2024-12-09 18:27:30,183 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits'.
2024-12-09 18:27:30,183 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits'.
2024-12-09 18:27:30,184 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 18:27:30,184 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 18:27:30,184 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 18:27:30,185 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits'.
2024-12-09 18:27:30,185 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits'.
2024-12-09 18:27:30,186 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 18:27:30,476 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:30,520 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 18:27:30,521 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 18:27:30,522 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 18:27:30,663 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:30,682 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits
2024-12-09 18:27:30,860 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 18:27:31,006 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:31,025 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits
2024-12-09 18:27:31,067 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:31,111 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:31,888 - stpipe.Detector1Pipeline.saturation - INFO - Detected 4958 saturated pixels
2024-12-09 18:27:31,899 - stpipe.Detector1Pipeline.saturation - INFO - Detected 1 A/D floor pixels
2024-12-09 18:27:31,904 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 18:27:32,061 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:32,061 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 18:27:32,209 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:32,227 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits
2024-12-09 18:27:32,394 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 18:27:32,541 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:32,599 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 18:27:32,599 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 18:27:32,600 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 18:27:32,600 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 18:27:32,601 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 18:27:32,601 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 18:27:32,602 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 18:27:32,602 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 18:27:34,088 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 18:27:34,237 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:34,256 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits
2024-12-09 18:27:34,309 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:34,309 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:34,310 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:34,310 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:34,311 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:34,312 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:34,312 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:34,737 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 18:27:34,888 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:34,889 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 18:27:35,029 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:35,048 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits
2024-12-09 18:27:36,075 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:36,113 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:36,124 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 18:27:36,125 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=187, nframes=1, groupgap=0
2024-12-09 18:27:36,709 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 18:27:36,855 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:36,856 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 18:27:36,993 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:37,002 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 18:27:37,003 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 18:27:37,013 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits
2024-12-09 18:27:37,036 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits
2024-12-09 18:27:37,224 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 18:27:37,251 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection
2024-12-09 18:27:39,759 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 18:27:42,233 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 83
2024-12-09 18:27:42,234 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 5.00852 sec
2024-12-09 18:27:42,285 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 5.282765
2024-12-09 18:27:42,288 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 18:27:42,436 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:42,464 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits
2024-12-09 18:27:42,465 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits
2024-12-09 18:27:42,493 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 18:27:42,494 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 18:27:42,657 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 18:27:45,285 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.623866319656372
2024-12-09 18:27:45,494 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 18:27:45,641 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:45,669 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:27:45,669 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:27:45,672 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:27:45,812 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 18:27:45,841 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:27:45,842 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:27:45,843 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:27:45,995 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rateints.fits
2024-12-09 18:27:45,996 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 18:27:45,996 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:27:46,148 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rate.fits
2024-12-09 18:27:46,149 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 18:27:46,233 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2024-12-09 18:27:46,236 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 18:27:46,248 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 18:27:46,265 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 18:27:46,266 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 18:27:46,267 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 18:27:46,268 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 18:27:46,269 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 18:27:46,270 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 18:27:46,271 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 18:27:46,272 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 18:27:46,273 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 18:27:46,274 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 18:27:46,275 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 18:27:46,276 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 18:27:46,276 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 18:27:46,277 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 18:27:46,278 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 18:27:46,279 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 18:27:46,280 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 18:27:46,282 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 18:27:46,283 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 18:27:46,444 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrcalong_uncal.fits',).
2024-12-09 18:27:46,464 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: False
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_intermediate_results: False
user_supplied_reffile: None
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
type: baseline
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 4.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: True
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
maximum_cores: '1'
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2024-12-09 18:27:46,536 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrcalong_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 18:27:46,540 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits'.
2024-12-09 18:27:46,540 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2024-12-09 18:27:46,541 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits'.
2024-12-09 18:27:46,541 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits'.
2024-12-09 18:27:46,541 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits'.
2024-12-09 18:27:46,542 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 18:27:46,542 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 18:27:46,543 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 18:27:46,543 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits'.
2024-12-09 18:27:46,543 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits'.
2024-12-09 18:27:46,545 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 18:27:46,825 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:46,870 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 18:27:46,870 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 18:27:46,872 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 18:27:47,015 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:47,033 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits
2024-12-09 18:27:47,225 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 18:27:47,370 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:47,389 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits
2024-12-09 18:27:47,431 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:47,476 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:48,253 - stpipe.Detector1Pipeline.saturation - INFO - Detected 26763 saturated pixels
2024-12-09 18:27:48,264 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2024-12-09 18:27:48,269 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 18:27:48,414 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:48,415 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 18:27:48,549 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:48,568 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits
2024-12-09 18:27:48,744 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 18:27:48,890 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:48,947 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 18:27:48,947 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 18:27:48,948 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 18:27:48,948 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 18:27:48,949 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 18:27:48,949 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 18:27:48,949 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 18:27:48,950 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 18:27:50,392 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 18:27:50,540 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:50,559 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits
2024-12-09 18:27:50,605 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:50,605 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:50,606 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:50,607 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:50,607 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:50,608 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:50,608 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 18:27:51,081 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 18:27:51,226 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:51,227 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 18:27:51,362 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:51,381 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits
2024-12-09 18:27:52,834 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 18:27:52,835 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=130, nframes=1, groupgap=0
2024-12-09 18:27:53,280 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 18:27:53,428 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:53,429 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 18:27:53,570 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:27:53,579 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 18:27:53,579 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 18:27:53,590 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2024-12-09 18:27:53,613 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits
2024-12-09 18:27:53,797 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 18:27:53,823 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection
2024-12-09 18:27:57,087 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 18:28:07,681 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 201
2024-12-09 18:28:07,682 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 13.8845 sec
2024-12-09 18:28:07,736 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 14.156854
2024-12-09 18:28:07,738 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 18:28:07,893 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:28:07,921 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits
2024-12-09 18:28:07,921 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2024-12-09 18:28:07,949 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 18:28:07,950 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 18:28:08,198 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 18:28:10,802 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.600226640701294
2024-12-09 18:28:11,012 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 18:28:11,162 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:28:11,190 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:28:11,190 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:28:11,192 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:28:11,338 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 18:28:11,366 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 18:28:11,367 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 18:28:11,369 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 18:28:11,520 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rateints.fits
2024-12-09 18:28:11,521 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 18:28:11,522 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:28:11,666 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rate.fits
2024-12-09 18:28:11,666 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
# Print out the time benchmark
time1 = time.perf_counter()
print(f"Runtime so far: {time1 - time0:0.0f} seconds")
print(f"Runtime for Detector1: {time1 - time_det1:0.0f} seconds")
Runtime so far: 887 seconds
Runtime for Detector1: 858 seconds
Exploring the data#
Identify *_rate.fits files and verify which pipeline steps were run and
which calibration reference files were applied.
The header contains information about which calibration steps were completed and skipped and which reference files were used to process the data.
if dodet1:
# find rate files
rate_files = sorted(glob.glob(os.path.join(det1_dir, '*_rate.fits')))
# Read in file as datamodel
rate_f = datamodels.open(rate_files[0])
# Check which steps were run
rate_f.meta.cal_step.instance
{'charge_migration': 'SKIPPED',
'dark_sub': 'COMPLETE',
'dq_init': 'COMPLETE',
'gain_scale': 'SKIPPED',
'group_scale': 'SKIPPED',
'ipc': 'SKIPPED',
'jump': 'COMPLETE',
'linearity': 'COMPLETE',
'persistence': 'SKIPPED',
'ramp_fit': 'COMPLETE',
'refpix': 'COMPLETE',
'saturation': 'COMPLETE',
'superbias': 'COMPLETE'}
For this particular rate file, show which reference files were used to calibrate the dataset. Note that these files will be different for each NIRCam detector.
if dodet1:
rate_f.meta.ref_file.instance
{'crds': {'context_used': 'jwst_1293.pmap', 'sw_version': '12.0.8'},
'dark': {'name': 'crds://jwst_nircam_dark_0342.fits'},
'gain': {'name': 'crds://jwst_nircam_gain_0093.fits'},
'linearity': {'name': 'crds://jwst_nircam_linearity_0051.fits'},
'mask': {'name': 'crds://jwst_nircam_mask_0072.fits'},
'readnoise': {'name': 'crds://jwst_nircam_readnoise_0224.fits'},
'saturation': {'name': 'crds://jwst_nircam_saturation_0104.fits'},
'superbias': {'name': 'crds://jwst_nircam_superbias_0186.fits'}}
6. Image2 Pipeline#
In the Image2 stage of the pipeline,
calibrated unrectified data products are created (*_cal.fits or
*_calints.fits files, depending on whether the input files are
*_rate.fits or *_rateints.fits).
In this pipeline processing stage, the world coordinate system (WCS) is assigned, the data are flat fielded, and a photometric calibration is applied to convert from units of countrate (ADU/s) to surface brightness (MJy/sr).
By default, the background subtraction step
and the resampling step
are turned off for NIRCam. The background
subtraction is turned off since there is no background template for the
imaging mode and the local background is subtracted as part of the photometry
perfoemd in the source catalog step in the Image3 pipeline.
The
resampling step occurs during the Image3 stage by default.
While the
resampling step can be run on individual images in the Image2 stage, e.g.,
to prepare for generating a source catalog for each image, the default behavior
is to run the step only in the Image3 stage, where multiple images are
combined into a final mosaic after the outlier detection step
flags bad pixels.
To turn on the resampling step in the Image2 stage, uncomment the line in the
dicitionary below which sets image2dict['resample']['skip'] = False
time_image2 = time.perf_counter()
# Set up a dictionary to define how the Image2 pipeline should be configured.
# Boilerplate dictionary setup
image2dict = {}
image2dict['assign_wcs'], image2dict['flat_field'] = {}, {}
image2dict['photom'], image2dict['resample'] = {}, {}
# Overrides for whether or not certain steps should be skipped (example)
#image2dict['resample']['skip'] = False
# Overrides for various reference files
# Files should be in the base local directory or provide full path
#image2dict['assign_wcs']['override_distortion'] = 'myfile.asdf' # Spatial distortion (ASDF file)
#image2dict['assign_wcs']['override_filteroffset'] = 'myfile.asdf' # Imager filter offsets (ASDF file)
#image2dict['assign_wcs']['override_specwcs'] = 'myfile.asdf' # Spectral distortion (ASDF file)
#image2dict['assign_wcs']['override_wavelengthrange'] = 'myfile.asdf' # Wavelength channel mapping (ASDF file)
#image2dict['flat_field']['override_flat'] = 'myfile.fits' # Pixel flatfield
#image2dict['photom']['override_photom'] = 'myfile.fits' # Photometric calibration array
Find and sort all of the input files, ensuring use of absolute paths
sstring = os.path.join(det1_dir, 'jw*rate.fits') # Use files from the detector1 output folder
rate_files = sorted(glob.glob(sstring))
rate_files = [os.path.abspath(fname) for fname in rate_files]
print(f"Found {len(rate_files)} science files")
Found 6 science files
# List rate files
rate_files
['/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rate.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rate.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rate.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rate.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rate.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rate.fits']
Run the Image2 pipeline on all of the rate files, regardless of filter. Note that if you have exposures with multiple integrations and you wish to keep the integrations separate, you should call the pipeline on the *rateints.fits files, rather than the *rate.fits files.
# Run Image2 stage of pipeline, specifying:
# output directory to save *_cal.fits files
# save_results flag set to True so the rate files are saved
if doimage2:
for rate in rate_files:
cal_result = Image2Pipeline.call(rate, output_dir=image2_dir, steps=image2dict, save_results=True)
else:
print("Skipping Image2 processing.")
2024-12-09 18:28:11,810 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf 1.1 K bytes (1 / 1 files) (0 / 1.1 K bytes)
2024-12-09 18:28:12,054 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 18:28:12,067 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 18:28:12,068 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 18:28:12,069 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 18:28:12,070 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 18:28:12,071 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 18:28:12,072 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 18:28:12,234 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rate.fits',).
2024-12-09 18:28:12,242 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
steps:
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_combined_background: False
sigma: 3.0
maxiters: None
wfss_mmag_extract: None
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
mrs_time_correction: True
resample:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: exptime
output_shape: None
crpix: None
crval: None
rotation: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
allowed_memory: None
in_memory: True
2024-12-09 18:28:12,292 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca2_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 18:28:12,296 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits 16.8 M bytes (1 / 5 files) (0 / 67.2 M bytes)
2024-12-09 18:28:13,575 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0268.asdf 14.3 K bytes (2 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 18:28:13,893 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf 11.4 K bytes (3 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 18:28:14,151 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits 50.4 M bytes (4 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 18:28:16,317 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits 23.0 K bytes (5 / 5 files) (67.2 M / 67.2 M bytes)
2024-12-09 18:28:16,645 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits'.
2024-12-09 18:28:16,646 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 18:28:16,646 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 18:28:16,647 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 18:28:16,647 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 18:28:16,647 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0268.asdf'.
2024-12-09 18:28:16,647 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 18:28:16,648 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf'.
2024-12-09 18:28:16,649 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits'.
2024-12-09 18:28:16,649 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 18:28:16,650 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 18:28:16,650 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 18:28:16,650 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 18:28:16,650 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 18:28:16,651 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 18:28:16,651 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 18:28:16,651 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits'.
2024-12-09 18:28:16,652 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 18:28:16,652 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 18:28:16,652 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 18:28:16,652 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 18:28:16,653 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 18:28:16,654 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 18:28:16,654 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2
2024-12-09 18:28:16,655 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rate.fits ...
2024-12-09 18:28:16,851 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_image2pipeline.fits>,).
2024-12-09 18:28:16,972 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 18:28:16,973 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:28:16,973 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:17,110 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS 274.711062770 -13.874490965 274.729108941 -13.875063132 274.729787198 -13.857677823 274.711707966 -13.856914583
2024-12-09 18:28:17,111 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS 274.711062770 -13.874490965 274.729108941 -13.875063132 274.729787198 -13.857677823 274.711707966 -13.856914583
2024-12-09 18:28:17,112 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 18:28:17,114 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:28:17,114 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use FITS instead.
2024-12-09 18:28:17,115 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:28:17,115 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:17,116 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:28:17,116 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use to_string() instead.
2024-12-09 18:28:17,117 - stpipe.Image2Pipeline.assign_wcs - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:28:17,117 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:17,158 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 18:28:17,308 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_image2pipeline.fits>,).
2024-12-09 18:28:17,389 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits
2024-12-09 18:28:17,390 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 18:28:17,390 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 18:28:17,391 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 18:28:17,505 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 18:28:17,655 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_image2pipeline.fits>,).
2024-12-09 18:28:17,679 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits
2024-12-09 18:28:17,679 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits
2024-12-09 18:28:17,706 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 18:28:17,706 - stpipe.Image2Pipeline.photom - INFO - detector: NRCA2
2024-12-09 18:28:17,707 - stpipe.Image2Pipeline.photom - INFO - exp_type: NRC_IMAGE
2024-12-09 18:28:17,707 - stpipe.Image2Pipeline.photom - INFO - filter: F200W
2024-12-09 18:28:17,707 - stpipe.Image2Pipeline.photom - INFO - pupil: CLEAR
2024-12-09 18:28:17,742 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 18:28:17,743 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 18:28:17,744 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 2.1
2024-12-09 18:28:17,761 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 18:28:17,912 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_image2pipeline.fits>,).
2024-12-09 18:28:17,929 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 18:28:17,930 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:28:17,930 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 18:28:17,931 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 18:28:17,931 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:28:17,958 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.030776005311869827 arcsec.
2024-12-09 18:28:18,012 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 18:28:19,873 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 18:28:20,695 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 18:28:22,694 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 18:28:25,596 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 18:28:28,488 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 18:28:29,398 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 274.711013782 -13.874493489 274.729096668 -13.875161576 274.729784767 -13.857589202 274.711703249 -13.856921167
2024-12-09 18:28:29,614 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_i2d.fits
2024-12-09 18:28:29,614 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 18:28:29,615 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2
2024-12-09 18:28:29,616 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 18:28:29,616 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:28:29,870 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_cal.fits
2024-12-09 18:28:29,871 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 18:28:29,923 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 18:28:29,936 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 18:28:29,937 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 18:28:29,938 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 18:28:29,939 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 18:28:29,940 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 18:28:29,941 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 18:28:30,100 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rate.fits',).
2024-12-09 18:28:30,108 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
steps:
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_combined_background: False
sigma: 3.0
maxiters: None
wfss_mmag_extract: None
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
mrs_time_correction: True
resample:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: exptime
output_shape: None
crpix: None
crval: None
rotation: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
allowed_memory: None
in_memory: True
2024-12-09 18:28:30,161 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca4_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 18:28:30,166 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits 16.8 M bytes (1 / 4 files) (0 / 67.2 M bytes)
2024-12-09 18:28:31,469 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0313.asdf 14.3 K bytes (2 / 4 files) (16.8 M / 67.2 M bytes)
2024-12-09 18:28:31,765 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits 50.4 M bytes (3 / 4 files) (16.8 M / 67.2 M bytes)
2024-12-09 18:28:33,925 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits 23.0 K bytes (4 / 4 files) (67.2 M / 67.2 M bytes)
2024-12-09 18:28:34,242 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits'.
2024-12-09 18:28:34,243 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 18:28:34,243 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 18:28:34,244 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 18:28:34,244 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 18:28:34,244 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0313.asdf'.
2024-12-09 18:28:34,245 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 18:28:34,245 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf'.
2024-12-09 18:28:34,245 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits'.
2024-12-09 18:28:34,246 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 18:28:34,247 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 18:28:34,247 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 18:28:34,247 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 18:28:34,248 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 18:28:34,248 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 18:28:34,248 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 18:28:34,248 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits'.
2024-12-09 18:28:34,249 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 18:28:34,250 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 18:28:34,250 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 18:28:34,251 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 18:28:34,252 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 18:28:34,253 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 18:28:34,253 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4
2024-12-09 18:28:34,253 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rate.fits ...
2024-12-09 18:28:34,454 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_image2pipeline.fits>,).
2024-12-09 18:28:34,572 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 18:28:34,572 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:28:34,573 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:34,703 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS 274.711757205 -13.855753713 274.729845583 -13.856419657 274.730660309 -13.838961453 274.712484599 -13.838101450
2024-12-09 18:28:34,703 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS 274.711757205 -13.855753713 274.729845583 -13.856419657 274.730660309 -13.838961453 274.712484599 -13.838101450
2024-12-09 18:28:34,704 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 18:28:34,706 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:28:34,707 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use FITS instead.
2024-12-09 18:28:34,707 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:28:34,707 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:34,708 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:28:34,708 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use to_string() instead.
2024-12-09 18:28:34,709 - stpipe.Image2Pipeline.assign_wcs - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:28:34,709 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:34,731 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2906: UserWarning: Double sampling check FAILED: Sampling may be too coarse for the distortion model being fitted.
2024-12-09 18:28:34,731 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:28:34,732 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:34,755 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 18:28:34,908 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_image2pipeline.fits>,).
2024-12-09 18:28:34,987 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits
2024-12-09 18:28:34,988 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 18:28:34,988 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 18:28:34,988 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 18:28:35,103 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 18:28:35,261 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_image2pipeline.fits>,).
2024-12-09 18:28:35,286 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits
2024-12-09 18:28:35,287 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits
2024-12-09 18:28:35,316 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 18:28:35,316 - stpipe.Image2Pipeline.photom - INFO - detector: NRCA4
2024-12-09 18:28:35,317 - stpipe.Image2Pipeline.photom - INFO - exp_type: NRC_IMAGE
2024-12-09 18:28:35,318 - stpipe.Image2Pipeline.photom - INFO - filter: F200W
2024-12-09 18:28:35,318 - stpipe.Image2Pipeline.photom - INFO - pupil: CLEAR
2024-12-09 18:28:35,350 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 18:28:35,351 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 18:28:35,352 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 1.829
2024-12-09 18:28:35,369 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 18:28:35,542 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_image2pipeline.fits>,).
2024-12-09 18:28:35,560 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 18:28:35,560 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:28:35,561 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 18:28:35,562 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 18:28:35,562 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:28:35,588 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.030907888443991595 arcsec.
2024-12-09 18:28:35,643 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 18:28:37,689 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 18:28:38,511 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 18:28:40,546 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 18:28:43,451 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 18:28:46,357 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 18:28:47,277 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 274.711689861 -13.855755202 274.729871311 -13.856521792 274.730660128 -13.838869390 274.712480056 -13.838102858
2024-12-09 18:28:47,492 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_i2d.fits
2024-12-09 18:28:47,492 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 18:28:47,493 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4
2024-12-09 18:28:47,494 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 18:28:47,494 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:28:47,751 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_cal.fits
2024-12-09 18:28:47,752 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 18:28:47,805 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 18:28:47,818 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 18:28:47,819 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 18:28:47,821 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 18:28:47,822 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 18:28:47,823 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 18:28:47,824 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 18:28:47,985 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rate.fits',).
2024-12-09 18:28:47,993 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
steps:
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_combined_background: False
sigma: 3.0
maxiters: None
wfss_mmag_extract: None
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
mrs_time_correction: True
resample:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: exptime
output_shape: None
crpix: None
crval: None
rotation: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
allowed_memory: None
in_memory: True
2024-12-09 18:28:48,047 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrcalong_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 18:28:48,051 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits 16.8 M bytes (1 / 5 files) (0 / 67.2 M bytes)
2024-12-09 18:28:49,357 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0277.asdf 12.7 K bytes (2 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 18:28:49,650 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf 11.4 K bytes (3 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 18:28:49,884 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits 50.4 M bytes (4 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 18:28:52,277 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits 23.0 K bytes (5 / 5 files) (67.2 M / 67.2 M bytes)
2024-12-09 18:28:52,576 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits'.
2024-12-09 18:28:52,576 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 18:28:52,577 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 18:28:52,577 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 18:28:52,578 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 18:28:52,578 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0277.asdf'.
2024-12-09 18:28:52,579 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 18:28:52,580 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf'.
2024-12-09 18:28:52,580 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits'.
2024-12-09 18:28:52,581 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 18:28:52,581 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 18:28:52,582 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 18:28:52,583 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 18:28:52,583 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 18:28:52,584 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 18:28:52,584 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 18:28:52,585 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits'.
2024-12-09 18:28:52,585 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 18:28:52,586 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 18:28:52,586 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 18:28:52,586 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 18:28:52,587 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 18:28:52,588 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 18:28:52,588 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong
2024-12-09 18:28:52,589 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rate.fits ...
2024-12-09 18:28:52,791 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_image2pipeline.fits>,).
2024-12-09 18:28:52,909 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 18:28:52,909 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:28:52,910 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:53,039 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS 274.692000981 -13.873604403 274.728935032 -13.874635765 274.730488193 -13.839266605 274.693322826 -13.837371261
2024-12-09 18:28:53,040 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS 274.692000981 -13.873604403 274.728935032 -13.874635765 274.730488193 -13.839266605 274.693322826 -13.837371261
2024-12-09 18:28:53,040 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 18:28:53,042 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:28:53,043 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use FITS instead.
2024-12-09 18:28:53,043 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:28:53,044 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:53,044 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:28:53,044 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use to_string() instead.
2024-12-09 18:28:53,045 - stpipe.Image2Pipeline.assign_wcs - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:28:53,045 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:28:53,094 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 18:28:53,248 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_image2pipeline.fits>,).
2024-12-09 18:28:53,328 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits
2024-12-09 18:28:53,328 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 18:28:53,329 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 18:28:53,330 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 18:28:53,444 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 18:28:53,599 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_image2pipeline.fits>,).
2024-12-09 18:28:53,623 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits
2024-12-09 18:28:53,624 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits
2024-12-09 18:28:53,652 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 18:28:53,653 - stpipe.Image2Pipeline.photom - INFO - detector: NRCALONG
2024-12-09 18:28:53,654 - stpipe.Image2Pipeline.photom - INFO - exp_type: NRC_IMAGE
2024-12-09 18:28:53,654 - stpipe.Image2Pipeline.photom - INFO - filter: F444W
2024-12-09 18:28:53,654 - stpipe.Image2Pipeline.photom - INFO - pupil: CLEAR
2024-12-09 18:28:53,687 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 18:28:53,687 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 18:28:53,689 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 0.402
2024-12-09 18:28:53,706 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 18:28:53,861 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_image2pipeline.fits>,).
2024-12-09 18:28:53,878 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 18:28:53,879 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:28:53,879 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 18:28:53,880 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 18:28:53,881 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:28:53,907 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.0629062603885582 arcsec.
2024-12-09 18:28:53,962 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 18:28:56,002 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 18:28:56,830 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 18:28:58,883 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 18:29:01,809 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 18:29:04,734 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 18:29:05,663 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 274.691795163 -13.873605296 274.728986649 -13.875071623 274.730499024 -13.838842900 274.693313328 -13.837376803
2024-12-09 18:29:05,882 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_i2d.fits
2024-12-09 18:29:05,883 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 18:29:05,883 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong
2024-12-09 18:29:05,884 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 18:29:05,885 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:29:06,144 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_cal.fits
2024-12-09 18:29:06,144 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 18:29:06,198 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 18:29:06,211 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 18:29:06,212 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 18:29:06,213 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 18:29:06,214 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 18:29:06,215 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 18:29:06,217 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 18:29:06,378 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rate.fits',).
2024-12-09 18:29:06,386 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
steps:
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_combined_background: False
sigma: 3.0
maxiters: None
wfss_mmag_extract: None
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
mrs_time_correction: True
resample:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: exptime
output_shape: None
crpix: None
crval: None
rotation: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
allowed_memory: None
in_memory: True
2024-12-09 18:29:06,440 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrca2_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 18:29:06,444 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits'.
2024-12-09 18:29:06,445 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 18:29:06,445 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 18:29:06,446 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 18:29:06,446 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 18:29:06,446 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0268.asdf'.
2024-12-09 18:29:06,447 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 18:29:06,448 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf'.
2024-12-09 18:29:06,449 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits'.
2024-12-09 18:29:06,449 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 18:29:06,450 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 18:29:06,450 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 18:29:06,451 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 18:29:06,451 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 18:29:06,451 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 18:29:06,452 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 18:29:06,452 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits'.
2024-12-09 18:29:06,453 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 18:29:06,454 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 18:29:06,454 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 18:29:06,455 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 18:29:06,455 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 18:29:06,456 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 18:29:06,456 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2
2024-12-09 18:29:06,457 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rate.fits ...
2024-12-09 18:29:06,662 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_image2pipeline.fits>,).
2024-12-09 18:29:06,779 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 18:29:06,780 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:29:06,780 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:06,912 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS 274.714148074 -13.874666536 274.732194265 -13.875238512 274.732872327 -13.857853196 274.714793074 -13.857090147
2024-12-09 18:29:06,912 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS 274.714148074 -13.874666536 274.732194265 -13.875238512 274.732872327 -13.857853196 274.714793074 -13.857090147
2024-12-09 18:29:06,913 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 18:29:06,915 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:29:06,916 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use FITS instead.
2024-12-09 18:29:06,916 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:29:06,916 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:06,917 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:29:06,917 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use to_string() instead.
2024-12-09 18:29:06,918 - stpipe.Image2Pipeline.assign_wcs - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:29:06,918 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:06,958 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 18:29:07,114 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_image2pipeline.fits>,).
2024-12-09 18:29:07,191 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits
2024-12-09 18:29:07,192 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 18:29:07,192 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 18:29:07,193 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 18:29:07,307 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 18:29:07,464 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_image2pipeline.fits>,).
2024-12-09 18:29:07,488 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits
2024-12-09 18:29:07,489 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits
2024-12-09 18:29:07,517 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 18:29:07,518 - stpipe.Image2Pipeline.photom - INFO - detector: NRCA2
2024-12-09 18:29:07,519 - stpipe.Image2Pipeline.photom - INFO - exp_type: NRC_IMAGE
2024-12-09 18:29:07,519 - stpipe.Image2Pipeline.photom - INFO - filter: F200W
2024-12-09 18:29:07,519 - stpipe.Image2Pipeline.photom - INFO - pupil: CLEAR
2024-12-09 18:29:07,551 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 18:29:07,552 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 18:29:07,553 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 2.1
2024-12-09 18:29:07,570 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 18:29:07,727 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_image2pipeline.fits>,).
2024-12-09 18:29:07,744 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 18:29:07,745 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:29:07,746 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 18:29:07,746 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 18:29:07,747 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:29:07,774 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.030776005265217777 arcsec.
2024-12-09 18:29:07,828 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 18:29:09,861 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 18:29:10,683 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 18:29:12,727 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 18:29:15,663 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 18:29:18,574 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 18:29:19,496 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 274.714099086 -13.874669061 274.732181993 -13.875336956 274.732869896 -13.857764576 274.714788356 -13.857096731
2024-12-09 18:29:19,712 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_i2d.fits
2024-12-09 18:29:19,713 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 18:29:19,714 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2
2024-12-09 18:29:19,714 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 18:29:19,715 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:29:19,971 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_cal.fits
2024-12-09 18:29:19,971 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 18:29:20,025 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 18:29:20,038 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 18:29:20,039 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 18:29:20,041 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 18:29:20,042 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 18:29:20,043 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 18:29:20,044 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 18:29:20,204 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rate.fits',).
2024-12-09 18:29:20,213 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
steps:
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_combined_background: False
sigma: 3.0
maxiters: None
wfss_mmag_extract: None
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
mrs_time_correction: True
resample:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: exptime
output_shape: None
crpix: None
crval: None
rotation: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
allowed_memory: None
in_memory: True
2024-12-09 18:29:20,267 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrca4_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 18:29:20,272 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits'.
2024-12-09 18:29:20,272 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 18:29:20,273 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 18:29:20,273 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 18:29:20,274 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 18:29:20,275 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0313.asdf'.
2024-12-09 18:29:20,275 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 18:29:20,276 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf'.
2024-12-09 18:29:20,276 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits'.
2024-12-09 18:29:20,277 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 18:29:20,277 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 18:29:20,278 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 18:29:20,278 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 18:29:20,278 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 18:29:20,279 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 18:29:20,279 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 18:29:20,280 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits'.
2024-12-09 18:29:20,280 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 18:29:20,280 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 18:29:20,281 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 18:29:20,281 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 18:29:20,282 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 18:29:20,282 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 18:29:20,283 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4
2024-12-09 18:29:20,283 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rate.fits ...
2024-12-09 18:29:20,490 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_image2pipeline.fits>,).
2024-12-09 18:29:20,608 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 18:29:20,609 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:29:20,609 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:20,741 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS 274.714842299 -13.855929276 274.732930699 -13.856595030 274.733745230 -13.839136817 274.715569496 -13.838277006
2024-12-09 18:29:20,741 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS 274.714842299 -13.855929276 274.732930699 -13.856595030 274.733745230 -13.839136817 274.715569496 -13.838277006
2024-12-09 18:29:20,742 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 18:29:20,744 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:29:20,745 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use FITS instead.
2024-12-09 18:29:20,745 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:29:20,746 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:20,746 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:29:20,747 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use to_string() instead.
2024-12-09 18:29:20,747 - stpipe.Image2Pipeline.assign_wcs - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:29:20,748 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:20,769 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2906: UserWarning: Double sampling check FAILED: Sampling may be too coarse for the distortion model being fitted.
2024-12-09 18:29:20,770 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:29:20,770 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:20,793 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 18:29:20,958 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_image2pipeline.fits>,).
2024-12-09 18:29:21,034 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits
2024-12-09 18:29:21,035 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 18:29:21,035 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 18:29:21,035 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 18:29:21,150 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 18:29:21,309 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_image2pipeline.fits>,).
2024-12-09 18:29:21,333 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits
2024-12-09 18:29:21,333 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits
2024-12-09 18:29:21,362 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 18:29:21,362 - stpipe.Image2Pipeline.photom - INFO - detector: NRCA4
2024-12-09 18:29:21,363 - stpipe.Image2Pipeline.photom - INFO - exp_type: NRC_IMAGE
2024-12-09 18:29:21,363 - stpipe.Image2Pipeline.photom - INFO - filter: F200W
2024-12-09 18:29:21,364 - stpipe.Image2Pipeline.photom - INFO - pupil: CLEAR
2024-12-09 18:29:21,395 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 18:29:21,396 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 18:29:21,397 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 1.829
2024-12-09 18:29:21,414 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 18:29:21,573 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_image2pipeline.fits>,).
2024-12-09 18:29:21,591 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 18:29:21,592 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:29:21,592 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 18:29:21,593 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 18:29:21,593 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:29:21,620 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.030907888553463565 arcsec.
2024-12-09 18:29:21,675 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 18:29:23,705 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 18:29:24,526 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 18:29:26,571 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 18:29:29,482 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 18:29:32,391 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 18:29:33,311 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 274.714774955 -13.855930766 274.732956428 -13.856697164 274.733745047 -13.839044754 274.715564953 -13.838278414
2024-12-09 18:29:33,528 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_i2d.fits
2024-12-09 18:29:33,528 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 18:29:33,529 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4
2024-12-09 18:29:33,530 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 18:29:33,530 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:29:33,789 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_cal.fits
2024-12-09 18:29:33,789 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 18:29:33,843 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 18:29:33,856 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 18:29:33,857 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 18:29:33,858 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 18:29:33,859 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 18:29:33,860 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 18:29:33,862 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 18:29:34,031 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rate.fits',).
2024-12-09 18:29:34,038 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
steps:
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_combined_background: False
sigma: 3.0
maxiters: None
wfss_mmag_extract: None
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
mrs_time_correction: True
resample:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: exptime
output_shape: None
crpix: None
crval: None
rotation: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
allowed_memory: None
in_memory: True
2024-12-09 18:29:34,092 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrcalong_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 18:29:34,097 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits'.
2024-12-09 18:29:34,097 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 18:29:34,098 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 18:29:34,098 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 18:29:34,098 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 18:29:34,099 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0277.asdf'.
2024-12-09 18:29:34,099 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 18:29:34,100 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf'.
2024-12-09 18:29:34,101 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits'.
2024-12-09 18:29:34,102 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 18:29:34,102 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 18:29:34,102 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 18:29:34,103 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 18:29:34,103 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 18:29:34,104 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 18:29:34,104 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 18:29:34,104 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits'.
2024-12-09 18:29:34,105 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 18:29:34,105 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 18:29:34,106 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 18:29:34,106 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 18:29:34,106 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 18:29:34,107 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 18:29:34,108 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong
2024-12-09 18:29:34,108 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rate.fits ...
2024-12-09 18:29:34,315 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_image2pipeline.fits>,).
2024-12-09 18:29:34,433 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 18:29:34,433 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:29:34,434 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:34,564 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS 274.695086261 -13.873780175 274.732020351 -13.874811147 274.733573116 -13.839441971 274.696407700 -13.837547020
2024-12-09 18:29:34,565 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS 274.695086261 -13.873780175 274.732020351 -13.874811147 274.733573116 -13.839441971 274.696407700 -13.837547020
2024-12-09 18:29:34,565 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 18:29:34,567 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:29:34,568 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use FITS instead.
2024-12-09 18:29:34,568 - stpipe.Image2Pipeline.assign_wcs - WARNING - warnings.warn(
2024-12-09 18:29:34,569 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:34,569 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:29:34,569 - stpipe.Image2Pipeline.assign_wcs - WARNING - Use to_string() instead.
2024-12-09 18:29:34,570 - stpipe.Image2Pipeline.assign_wcs - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:29:34,570 - stpipe.Image2Pipeline.assign_wcs - WARNING -
2024-12-09 18:29:34,619 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 18:29:34,777 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_image2pipeline.fits>,).
2024-12-09 18:29:34,856 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits
2024-12-09 18:29:34,857 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 18:29:34,857 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 18:29:34,858 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 18:29:34,972 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 18:29:35,133 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_image2pipeline.fits>,).
2024-12-09 18:29:35,157 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits
2024-12-09 18:29:35,158 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits
2024-12-09 18:29:35,186 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 18:29:35,186 - stpipe.Image2Pipeline.photom - INFO - detector: NRCALONG
2024-12-09 18:29:35,187 - stpipe.Image2Pipeline.photom - INFO - exp_type: NRC_IMAGE
2024-12-09 18:29:35,188 - stpipe.Image2Pipeline.photom - INFO - filter: F444W
2024-12-09 18:29:35,188 - stpipe.Image2Pipeline.photom - INFO - pupil: CLEAR
2024-12-09 18:29:35,219 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 18:29:35,220 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 18:29:35,221 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 0.402
2024-12-09 18:29:35,238 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 18:29:35,401 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_image2pipeline.fits>,).
2024-12-09 18:29:35,419 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 18:29:35,419 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:29:35,420 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 18:29:35,421 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 18:29:35,421 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:29:35,448 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.06290626050781634 arcsec.
2024-12-09 18:29:35,504 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 18:29:37,535 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 18:29:38,363 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 18:29:40,409 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 18:29:43,329 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 18:29:46,254 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 18:29:47,181 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 274.694880443 -13.873781071 274.732071973 -13.875247004 274.733583943 -13.839018266 274.696398203 -13.837552561
2024-12-09 18:29:47,401 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_i2d.fits
2024-12-09 18:29:47,401 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 18:29:47,402 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong
2024-12-09 18:29:47,403 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 18:29:47,403 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 18:29:47,662 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_cal.fits
2024-12-09 18:29:47,663 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
# Print out the time benchmark
time1 = time.perf_counter()
print(f"Runtime so far: {time1 - time0:0.0f} seconds")
print(f"Runtime for Image2: {time1 - time_image2:0.0f} seconds")
Runtime so far: 983 seconds
Runtime for Image2: 96 seconds
Verify which pipeline steps were run#
if doimage2:
# Identify *_cal.fits file products
cal_files = sorted(glob.glob(os.path.join(image2_dir, '*_cal.fits')))
# Select first file to gather information
cal_f = datamodels.open(cal_files[0])
# Check which steps were run:
cal_f.meta.cal_step.instance
{'assign_wcs': 'COMPLETE',
'charge_migration': 'SKIPPED',
'dark_sub': 'COMPLETE',
'dq_init': 'COMPLETE',
'flat_field': 'COMPLETE',
'gain_scale': 'SKIPPED',
'group_scale': 'SKIPPED',
'ipc': 'SKIPPED',
'jump': 'COMPLETE',
'linearity': 'COMPLETE',
'persistence': 'SKIPPED',
'photom': 'COMPLETE',
'ramp_fit': 'COMPLETE',
'refpix': 'COMPLETE',
'saturation': 'COMPLETE',
'superbias': 'COMPLETE'}
Check which reference files were used to calibrate the first file. Some of these will be detector-dependent.
if doimage2:
cal_f.meta.ref_file.instance
{'area': {'name': 'crds://jwst_nircam_area_0263.fits'},
'camera': {'name': 'N/A'},
'collimator': {'name': 'N/A'},
'crds': {'context_used': 'jwst_1293.pmap', 'sw_version': '12.0.8'},
'dark': {'name': 'crds://jwst_nircam_dark_0342.fits'},
'dflat': {'name': 'N/A'},
'disperser': {'name': 'N/A'},
'distortion': {'name': 'crds://jwst_nircam_distortion_0268.asdf'},
'fflat': {'name': 'N/A'},
'filteroffset': {'name': 'crds://jwst_nircam_filteroffset_0005.asdf'},
'flat': {'name': 'crds://jwst_nircam_flat_0654.fits'},
'fore': {'name': 'N/A'},
'fpa': {'name': 'N/A'},
'gain': {'name': 'crds://jwst_nircam_gain_0093.fits'},
'ifufore': {'name': 'N/A'},
'ifupost': {'name': 'N/A'},
'ifuslicer': {'name': 'N/A'},
'linearity': {'name': 'crds://jwst_nircam_linearity_0051.fits'},
'mask': {'name': 'crds://jwst_nircam_mask_0072.fits'},
'msa': {'name': 'N/A'},
'ote': {'name': 'N/A'},
'photom': {'name': 'crds://jwst_nircam_photom_0152.fits'},
'readnoise': {'name': 'crds://jwst_nircam_readnoise_0224.fits'},
'regions': {'name': 'N/A'},
'saturation': {'name': 'crds://jwst_nircam_saturation_0104.fits'},
'sflat': {'name': 'N/A'},
'specwcs': {'name': 'N/A'},
'superbias': {'name': 'crds://jwst_nircam_superbias_0186.fits'},
'wavelengthrange': {'name': 'N/A'}}
7. Image3 Pipeline#
In the Image3 stage of the pipeline, the individual *_cal.fits files for each filter are combined to one single distortion corrected image. Unlike the previous stages, we must run the Image3 stage separately for the files from each filter as well as channel (i.e. shortwave vs longwave).
First, we need to create Associations, to inform the pipeline which files are linked together for each filter.
By default, the Image3 stage of the pipeline performs the following steps on NIRCam data:
tweakreg - creates source catalogs of pointlike sources for each input image. The source catalog for each input image is compared to each other to derive coordinate transforms to align the images relative to each other.
tweakreghas many input parameters that can be adjusted to improve the image alignment in cases where the default values do not perform well.One tweakreg parameter that is not set by default but can be very useful is
abs_refcat. When this parameter is set toGAIADR3, the tweakreg step performs an absolute astrometric correction of the data using the GAIA data release 3 catalog. In cases where multiple unsaturated GAIA stars are present in the input images, this can improve the absolute astrometric alignment. However, in sparse or very crowded fields, this can potentially result in poor performance, so users are encouraged to check astrometric accuracy and revisit this step if necessary.As of pipeline version 1.14.0, the default source finding algorithm in the
tweakreg stepisIRAFStarFinder. Other options includeDAOStarFinder, whose results are not as good in cases where the PSF is undersampled, such as in the blue filters of the NIRCam shortwave channel. Finally photutils segmentation SourceFinder, which does not assume sources are point-like.
skymatch - measures the background level from the sky to use as input into the subsequent
outlier detectionandresamplesteps.outlier detection - flags any remaining cosmic rays, bad pixels, or other artifacts not already flagged during the
Detector1stage of the pipeline, using all input images to create a median image so that outliers in individual images can be identified.resample - resamples each input image based on its WCS and distortion information and creates a single undistorted image.
source catalog - creates a catalog of detected sources along with photometric results and morphologies (i.e., point-like vs extended). These catalogs are generally useful for quick checks, but optimization is likely needed for specific science cases. Users may wish to experiment with changing the
snr_thresholdanddeblendoptions. Modifications to the following parameters will not significantly improve data quality and it is advised to keep them at their default values:aperture_ee1,aperture_ee2,aperture_ee3,ci1_star_threshold,ci2_star_threshold.
time_image3 = time.perf_counter()
# Set up a dictionary to define how the Image3 pipeline should be configured
# Boilerplate dictionary setup
image3dict = {}
image3dict['assign_mtwcs'], image3dict['tweakreg'], image3dict['skymatch'] = {}, {}, {}
image3dict['outlier_detection'], image3dict['resample'], image3dict['source_catalog'] = {}, {}, {}
# Overrides for whether or not certain steps should be skipped (example)
#image3dict['outlier_detection']['skip'] = True
# Overrides for various reference files
# Files should be in the base local directory or provide full path
#image3dict['source_catalog']['override_apcorr'] = 'myfile.fits' # Aperture correction parameters
#image3dict['source_catalog']['override_abvegaoffset'] = 'myfile.asdf' # Data to convert from AB to Vega magnitudes (ASDF file)
# Turn on alignment to GAIA in the tweakreg step
# For data such as these demo data, where there are some heavily saturated stars in the field
# of view, alignment to GAIA sometimes does not work well due to tweakreg doing a poor job
# finding the centroids of the sources.
#image3dict['tweakreg']['abs_refcat'] = 'GAIADR3'
Find and sort all of the input files, ensuring use of absolute paths. Keep files for the two filters separated.
# Science Files need the cal.fits files
sw_sstring = os.path.join(image2_dir, 'jw*nrc??_cal.fits') # shortwave files. Detectors a1-a4, b1-b4
lw_sstring = os.path.join(image2_dir, 'jw*nrc*long_cal.fits') # longwave files. Detectors along, blong
# Identify SW and LW cal files
sw_cal_files = sorted(glob.glob(sw_sstring))
lw_cal_files = sorted(glob.glob(lw_sstring))
# Expand the relative paths into absolute paths
sw_cal_files = [os.path.abspath(fname) for fname in sw_cal_files]
lw_cal_files = [os.path.abspath(fname) for fname in lw_cal_files]
print(f'Found {len(sw_cal_files)} shortwave science files to process')
print(f'Found {len(lw_cal_files)} longwave science files to process')
Found 4 shortwave science files to process
Found 2 longwave science files to process
Create Association File#
An association file lists the files to calibrate together in Stage3 of the pipeline. Note that association files are available for download from MAST, with filenames of *_asn.json. Here we show how to create an association file to point to the data products created in the steps above. This is useful in cases where you want to work with a set of data that is different than that in the association files from MAST.
Note that the output products will have a rootname that is specified by the product_name in the association file. For this tutorial, the rootnames of the output products will be image3_sw for filter F200W and image3_lw for filter F444W.
# List of data to use
print('List of SW cal files to use:')
sw_cal_files
print('\nList of LW cal files to use:')
lw_cal_files
List of SW cal files to use:
['/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_cal.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_cal.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_cal.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_cal.fits']
List of LW cal files to use:
['/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_cal.fits',
'/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_cal.fits']
# Create Level 3 Association for SW products
do_swimage3 = False
if doimage3:
if len(sw_cal_files) > 0:
# Only create an association file if there are SW data files to process
do_swimage3 = True
sw_product_name = 'image3_sw'
sw_association = asn_from_list.asn_from_list(sw_cal_files,
rule=DMS_Level3_Base,
product_name=sw_product_name)
sw_association.data['asn_type'] = 'image3'
program = datamodels.open(sw_cal_files[0]).meta.observation.program_number
sw_association.data['program'] = program
# Format association as .json file
sw_asn_filename, sw_serialized = sw_association.dump(format="json")
# Write out association file
sw_association_im3 = os.path.join(sci_dir, sw_asn_filename)
with open(sw_association_im3, "w") as fd:
fd.write(sw_serialized)
2024-12-09 18:29:47,948 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename. All input files should be in a single directory, so no path is needed.
2024-12-09 18:29:47,949 - stpipe - WARNING - warnings.warn(err_str, UserWarning)
2024-12-09 18:29:47,950 - stpipe - WARNING -
1533
# Create Level 3 Associations for LW products
do_lwimage3 = False
if doimage3:
if len(lw_cal_files) > 0:
# Only create an association file if there are SW data files to process
do_lwimage3 = True
lw_product_name = 'image3_lw'
lw_association = asn_from_list.asn_from_list(lw_cal_files,
rule=DMS_Level3_Base,
product_name=lw_product_name)
lw_association.data['asn_type'] = 'image3'
program = datamodels.open(lw_cal_files[0]).meta.observation.program_number
lw_association.data['program'] = program
# Format association as .json file
lw_asn_filename, lw_serialized = lw_association.dump(format="json")
# Write out association file. Note that you can use your own filename in
# place of lw_asn_filename and everything will still work.
lw_association_im3 = os.path.join(sci_dir, lw_asn_filename)
with open(lw_association_im3, "w") as fd:
fd.write(lw_serialized)
2024-12-09 18:29:48,078 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename. All input files should be in a single directory, so no path is needed.
2024-12-09 18:29:48,079 - stpipe - WARNING - warnings.warn(err_str, UserWarning)
2024-12-09 18:29:48,079 - stpipe - WARNING -
989
Run Image3 stage of the pipeline#
For each set of grouped exposures in an association file, the Image3 stage of the pipeline will produce:
a
*_crf.fitsfile produced by theoutlier_detectionstep, where theDQarray marks the pixels flagged as outliers.a final combined, rectified image with name
*_i2d.fits,a source catalog with name
*_cat.ecsv,a segmentation map file (
*_segm.fits) which has integer values at the pixel locations where a source is detected where the pixel values match the source ID number in the catalog.
Run Image3 on the LW data#
# Run Stage3 on the LW data
if doimage3 and do_lwimage3:
lw_i2d_result = Image3Pipeline.call(lw_association_im3, output_dir=image3_dir, steps=image3dict, save_results=True)
else:
print('Skipping Image3 LW processing')
2024-12-09 18:29:48,273 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-tweakregstep_0045.asdf 1.6 K bytes (1 / 1 files) (0 / 1.6 K bytes)
2024-12-09 18:29:48,517 - stpipe - INFO - PARS-TWEAKREGSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-tweakregstep_0045.asdf
2024-12-09 18:29:48,529 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 18:29:48,538 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-sourcecatalogstep_0001.asdf 992 bytes (1 / 1 files) (0 / 992 bytes)
2024-12-09 18:29:48,779 - stpipe - INFO - PARS-SOURCECATALOGSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-sourcecatalogstep_0001.asdf
2024-12-09 18:29:48,794 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created.
2024-12-09 18:29:48,795 - stpipe.Image3Pipeline.assign_mtwcs - INFO - AssignMTWcsStep instance created.
2024-12-09 18:29:48,797 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created.
2024-12-09 18:29:48,798 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created.
2024-12-09 18:29:48,800 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created.
2024-12-09 18:29:48,802 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 18:29:48,803 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created.
2024-12-09 18:29:48,981 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('./nrc_im_demo_data/Obs001/jw02739-a3001_image3_00009_asn.json',).
2024-12-09 18:29:48,994 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage3
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
steps:
assign_mtwcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: False
suffix: assign_mtwcs
search_output_file: True
input_dir: ''
tweakreg:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_catalogs: False
use_custom_catalogs: False
catalog_format: ecsv
catfile: ''
starfinder: iraf
snr_threshold: 10.0
bkg_boxsize: 400
kernel_fwhm: 2.302
minsep_fwhm: 0.0
sigma_radius: 1.5
sharplo: 0.2
sharphi: 1.0
roundlo: -1.0
roundhi: 1.0
brightest: 200
peakmax: None
npixels: 10
connectivity: '8'
nlevels: 32
contrast: 0.001
multithresh_mode: exponential
localbkg_width: 0
apermask_method: correct
kron_params: None
enforce_user_order: False
expand_refcat: False
minobj: 15
fitgeometry: shift
nclip: 3
sigma: 3.0
searchrad: 2.0
use2dhist: True
separation: 2.0
tolerance: 1.0
xoffset: 0.0
yoffset: 0.0
abs_refcat: ''
save_abs_catalog: False
abs_minobj: 15
abs_fitgeometry: rshift
abs_nclip: 3
abs_sigma: 3.0
abs_searchrad: 6.0
abs_use2dhist: True
abs_separation: 1.0
abs_tolerance: 0.7
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
skymatch:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
skymethod: match
match_down: True
subtract: False
stepsize: None
skystat: mode
dqbits: ~DO_NOT_USE+NON_SCIENCE
lower: None
upper: None
nclip: 5
lsigma: 4.0
usigma: 4.0
binwidth: 0.1
outlier_detection:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: False
input_dir: ''
weight_type: ivm
pixfrac: 1.0
kernel: square
fillval: INDEF
nlow: 0
nhigh: 0
maskpt: 0.7
snr: 5.0 4.0
scale: 1.2 0.7
backg: 0.0
kernel_size: 7 7
threshold_percent: 99.8
rolling_window_width: 25
ifu_second_check: False
save_intermediate_results: False
resample_data: True
good_bits: ~DO_NOT_USE
allowed_memory: None
in_memory: False
resample:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: exptime
output_shape: None
crpix: None
crval: None
rotation: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
allowed_memory: None
in_memory: True
source_catalog:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: cat
search_output_file: True
input_dir: ''
bkg_boxsize: 1000
kernel_fwhm: 2.302
snr_threshold: 3.0
npixels: 25
deblend: False
aperture_ee1: 30
aperture_ee2: 50
aperture_ee3: 70
ci1_star_threshold: 2.0
ci2_star_threshold: 1.8
2024-12-09 18:29:49,007 - stpipe.Image3Pipeline - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename. All input files should be in a single directory, so no path is needed.
2024-12-09 18:29:49,007 - stpipe.Image3Pipeline - WARNING - warnings.warn(err_str, UserWarning)
2024-12-09 18:29:49,008 - stpipe.Image3Pipeline - WARNING -
2024-12-09 18:29:49,141 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrcalong_cal.fits' reftypes = ['abvegaoffset', 'apcorr']
2024-12-09 18:29:49,145 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf 4.3 K bytes (1 / 2 files) (0 / 24.5 K bytes)
2024-12-09 18:29:49,373 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits 20.2 K bytes (2 / 2 files) (4.3 K / 24.5 K bytes)
2024-12-09 18:29:49,685 - stpipe.Image3Pipeline - INFO - Prefetch for ABVEGAOFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf'.
2024-12-09 18:29:49,686 - stpipe.Image3Pipeline - INFO - Prefetch for APCORR reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits'.
2024-12-09 18:29:49,687 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ...
2024-12-09 18:29:49,700 - stpipe.Image3Pipeline - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename. All input files should be in a single directory, so no path is needed.
2024-12-09 18:29:49,700 - stpipe.Image3Pipeline - WARNING - warnings.warn(err_str, UserWarning)
2024-12-09 18:29:49,701 - stpipe.Image3Pipeline - WARNING -
2024-12-09 18:29:50,134 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg running with args (<ModelContainer>,).
2024-12-09 18:29:51,641 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00001_nrcalong_cal.fits.
2024-12-09 18:29:53,169 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00002_nrcalong_cal.fits.
2024-12-09 18:29:53,193 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:29:53,193 - stpipe.Image3Pipeline.tweakreg - INFO - Number of image groups to be aligned: 2.
2024-12-09 18:29:53,194 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:29:53,194 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() started on 2024-12-09 18:29:53.194235
2024-12-09 18:29:53,195 - stpipe.Image3Pipeline.tweakreg - INFO - Version 0.8.9
2024-12-09 18:29:53,195 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:29:53,691 - stpipe.Image3Pipeline.tweakreg - INFO - Selected image 'GROUP ID: jw02739001002_02105_1' as reference image
2024-12-09 18:29:53,696 - stpipe.Image3Pipeline.tweakreg - INFO - Aligning image catalog 'GROUP ID: jw02739001002_02105_2' to the reference catalog.
2024-12-09 18:29:53,739 - stpipe.Image3Pipeline.tweakreg - INFO - Matching sources from 'jw02739001002_02105_00002_nrcalong_cal' catalog with sources from the reference 'jw02739001002_02105_00001_nrcalong_cal' catalog.
2024-12-09 18:29:53,740 - stpipe.Image3Pipeline.tweakreg - INFO - Computing initial guess for X and Y shifts...
2024-12-09 18:29:53,741 - stpipe.Image3Pipeline.tweakreg - INFO - Found initial X and Y shifts of 0.013, 0.013 (arcsec) with significance of 121 and 121 matches.
2024-12-09 18:29:53,742 - stpipe.Image3Pipeline.tweakreg - INFO - Found 117 matches for 'GROUP ID: jw02739001002_02105_2'...
2024-12-09 18:29:53,743 - stpipe.Image3Pipeline.tweakreg - INFO - Performing 'shift' fit
2024-12-09 18:29:53,745 - stpipe.Image3Pipeline.tweakreg - INFO - Computed 'shift' fit for GROUP ID: jw02739001002_02105_2:
2024-12-09 18:29:53,745 - stpipe.Image3Pipeline.tweakreg - INFO - XSH: -0.00263329 YSH: 0.00178288
2024-12-09 18:29:53,746 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:29:53,746 - stpipe.Image3Pipeline.tweakreg - INFO - FIT RMSE: 0.0408995 FIT MAE: 0.0127711
2024-12-09 18:29:53,746 - stpipe.Image3Pipeline.tweakreg - INFO - Final solution based on 113 objects.
2024-12-09 18:29:53,779 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:29:53,780 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() ended on 2024-12-09 18:29:53.779727
2024-12-09 18:29:53,780 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() TOTAL RUN TIME: 0:00:00.585492
2024-12-09 18:29:53,780 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:29:53,842 - stpipe.Image3Pipeline.tweakreg - INFO - Update S_REGION to POLYGON ICRS 274.695085507 -13.873779680 274.732019598 -13.874810652 274.733572363 -13.839441476 274.696406947 -13.837546524
2024-12-09 18:29:53,845 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:29:53,845 - stpipe.Image3Pipeline.tweakreg - WARNING - Use FITS instead.
2024-12-09 18:29:53,845 - stpipe.Image3Pipeline.tweakreg - WARNING - warnings.warn(
2024-12-09 18:29:53,846 - stpipe.Image3Pipeline.tweakreg - WARNING -
2024-12-09 18:29:53,846 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:29:53,846 - stpipe.Image3Pipeline.tweakreg - WARNING - Use to_string() instead.
2024-12-09 18:29:53,847 - stpipe.Image3Pipeline.tweakreg - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:29:53,847 - stpipe.Image3Pipeline.tweakreg - WARNING -
2024-12-09 18:29:53,909 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg done
2024-12-09 18:29:54,089 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch running with args (<ModelContainer>,).
2024-12-09 18:29:54,139 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:29:54,139 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() started on 2024-12-09 18:29:54.139191
2024-12-09 18:29:54,140 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:29:54,140 - stpipe.Image3Pipeline.skymatch - INFO - Sky computation method: 'match'
2024-12-09 18:29:54,140 - stpipe.Image3Pipeline.skymatch - INFO - Sky matching direction: DOWN
2024-12-09 18:29:54,141 - stpipe.Image3Pipeline.skymatch - INFO - Sky subtraction from image data: OFF
2024-12-09 18:29:54,141 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:29:54,141 - stpipe.Image3Pipeline.skymatch - INFO - ---- Computing differences in sky values in overlapping regions.
2024-12-09 18:29:55,135 - stpipe.Image3Pipeline.skymatch - INFO - * Image ID=jw02739001002_02105_00001_nrcalong_cal.fits. Sky background: 0
2024-12-09 18:29:55,135 - stpipe.Image3Pipeline.skymatch - INFO - * Image ID=jw02739001002_02105_00002_nrcalong_cal.fits. Sky background: 0.0721858
2024-12-09 18:29:55,136 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:29:55,136 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() ended on 2024-12-09 18:29:55.136397
2024-12-09 18:29:55,137 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() TOTAL RUN TIME: 0:00:00.997206
2024-12-09 18:29:55,137 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:29:55,142 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch done
2024-12-09 18:29:55,313 - stpipe.Image3Pipeline.outlier_detection - INFO - Step outlier_detection running with args (<ModelContainer>,).
2024-12-09 18:29:55,319 - stpipe.Image3Pipeline.outlier_detection - INFO - Performing outlier detection on 2 inputs
2024-12-09 18:29:55,320 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter kernel: square
2024-12-09 18:29:55,320 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:29:55,321 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter fillval: INDEF
2024-12-09 18:29:55,321 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter weight_type: ivm
2024-12-09 18:29:55,321 - stpipe.Image3Pipeline.outlier_detection - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:29:55,356 - stpipe.Image3Pipeline.outlier_detection - INFO - Computed output pixel scale: 0.0629062603885582 arcsec.
2024-12-09 18:29:55,412 - stpipe.Image3Pipeline.outlier_detection - INFO - 1 exposures to drizzle together
2024-12-09 18:29:57,300 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:29:58,285 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_outlier_i2d.fits
2024-12-09 18:29:58,291 - stpipe.Image3Pipeline.outlier_detection - INFO - 1 exposures to drizzle together
2024-12-09 18:30:00,721 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:01,706 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_outlier_i2d.fits
2024-12-09 18:30:01,825 - stpipe.Image3Pipeline.outlier_detection - INFO - Computing median
2024-12-09 18:30:06,426 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_outlier_i2d.fits
2024-12-09 18:30:06,439 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_outlier_i2d.fits
2024-12-09 18:30:06,442 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting median
2024-12-09 18:30:08,510 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2240, 2078)
2024-12-09 18:30:08,842 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_blot.fits
2024-12-09 18:30:11,528 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2240, 2078)
2024-12-09 18:30:11,881 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_blot.fits
2024-12-09 18:30:11,882 - stpipe.Image3Pipeline.outlier_detection - INFO - Flagging outliers
2024-12-09 18:30:12,187 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 484 (0.01%)
2024-12-09 18:30:12,541 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 186 (0.00%)
2024-12-09 18:30:12,568 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_blot.fits
2024-12-09 18:30:12,593 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_blot.fits
2024-12-09 18:30:12,854 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_crf.fits
2024-12-09 18:30:13,139 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_crf.fits
2024-12-09 18:30:13,139 - stpipe.Image3Pipeline.outlier_detection - INFO - Step outlier_detection done
2024-12-09 18:30:13,323 - stpipe.Image3Pipeline.resample - INFO - Step resample running with args (<ModelContainer>,).
2024-12-09 18:30:13,330 - stpipe.Image3Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 18:30:13,330 - stpipe.Image3Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:30:13,330 - stpipe.Image3Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 18:30:13,331 - stpipe.Image3Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 18:30:13,331 - stpipe.Image3Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:30:13,365 - stpipe.Image3Pipeline.resample - INFO - Computed output pixel scale: 0.0629062603885582 arcsec.
2024-12-09 18:30:13,419 - stpipe.Image3Pipeline.resample - INFO - Blending metadata for None
2024-12-09 18:30:13,961 - stpipe.Image3Pipeline.resample - INFO - Resampling science data
2024-12-09 18:30:15,948 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:19,323 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:20,156 - stpipe.Image3Pipeline.resample - INFO - Resampling variance components
2024-12-09 18:30:22,181 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:25,120 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:28,032 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:31,515 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:35,014 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:38,506 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 18:30:39,436 - stpipe.Image3Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 274.691792687 -13.873658631 274.732077488 -13.875246915 274.733591808 -13.838965804 274.693313288 -13.837377769
2024-12-09 18:30:39,969 - stpipe.Image3Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/image3_lw_i2d.fits
2024-12-09 18:30:39,969 - stpipe.Image3Pipeline.resample - INFO - Step resample done
2024-12-09 18:30:40,148 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog running with args (<ImageModel(2240, 2078) from image3_lw_i2d.fits>,).
2024-12-09 18:30:40,163 - stpipe.Image3Pipeline.source_catalog - INFO - Using APCORR reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits
2024-12-09 18:30:40,171 - stpipe.Image3Pipeline.source_catalog - INFO - Using ABVEGAOFFSET reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf
2024-12-09 18:30:40,172 - stpipe.Image3Pipeline.source_catalog - INFO - Instrument: NIRCAM
2024-12-09 18:30:40,172 - stpipe.Image3Pipeline.source_catalog - INFO - Detector: NRCALONG
2024-12-09 18:30:40,172 - stpipe.Image3Pipeline.source_catalog - INFO - Filter: F444W
2024-12-09 18:30:40,173 - stpipe.Image3Pipeline.source_catalog - INFO - Pupil: CLEAR
2024-12-09 18:30:40,173 - stpipe.Image3Pipeline.source_catalog - INFO - Subarray: FULL
2024-12-09 18:30:40,196 - stpipe.Image3Pipeline.source_catalog - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 18:30:40,196 - stpipe.Image3Pipeline.source_catalog - WARNING - warnings.warn(
2024-12-09 18:30:40,196 - stpipe.Image3Pipeline.source_catalog - WARNING -
2024-12-09 18:30:40,214 - stpipe.Image3Pipeline.source_catalog - INFO - AB to Vega magnitude offset 3.23800
2024-12-09 18:30:43,420 - stpipe.Image3Pipeline.source_catalog - INFO - Detected 1489 sources
2024-12-09 18:30:44,733 - stpipe.Image3Pipeline.source_catalog - INFO - Wrote source catalog: ./nrc_im_demo_data/Obs001/stage3/image3_lw_cat.ecsv
2024-12-09 18:30:44,872 - stpipe.Image3Pipeline.source_catalog - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/image3_lw_segm.fits
2024-12-09 18:30:44,873 - stpipe.Image3Pipeline.source_catalog - INFO - Wrote segmentation map: image3_lw_segm.fits
2024-12-09 18:30:44,874 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog done
2024-12-09 18:30:44,876 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline done
Some users wish to resample data from multiple filters onto the same WCS and pixel grid in order to create color images or help with subsequent analyses. In order to do that, we’ll save the gWCS from the *i2d.fits file created with the LW data above. The gWCS will be saved into an asdf file.
if doimage3 and do_lwimage3:
# First we identify the dataset and read it using datamodels.
lw_i2d_file = os.path.join(image3_dir, f'{lw_product_name}_i2d.fits')
lw_data = datamodels.open(lw_i2d_file)
# Pull out the resulting gWCS and save it in an asdf file
tree = {"wcs": lw_data.meta.wcs}
wcs_file = AsdfFile(tree)
gwcs_filename = os.path.join(image3_dir + 'lw_gwcs.asdf')
print(f'Saving gWCS into {gwcs_filename}')
wcs_file.write_to(gwcs_filename)
# Get the size of the mosaic image
ysize, xsize = lw_data.data.shape
Saving gWCS into ./nrc_im_demo_data/Obs001/stage3lw_gwcs.asdf
Run Image3 on the SW data#
Prepare to call the Image3 pipeline on the SW data. If you wish to resample the SW data onto the same pixel grid as the LW data above, uncomment the lines below. This will tell the resample step to use the gWCS and the array size from the LW data when resampling the SW data.
# Uncoment this cell in order to resample the SW data onto the same pixel grid as the LW data
#if doimage3:
# image3dict['resample']['output_wcs'] = gwcs_filename
# image3dict['resample']['output_shape'] = (xsize, ysize)
if doimage3 and do_swimage3:
sw_i2d_result = Image3Pipeline.call(sw_association_im3, output_dir=image3_dir, steps=image3dict, save_results=True)
else:
print('Skipping Image3 SW processing')
2024-12-09 18:30:45,212 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename. All input files should be in a single directory, so no path is needed.
2024-12-09 18:30:45,213 - stpipe - WARNING - warnings.warn(err_str, UserWarning)
2024-12-09 18:30:45,213 - stpipe - WARNING -
2024-12-09 18:30:45,366 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-tweakregstep_0036.asdf 1.6 K bytes (1 / 1 files) (0 / 1.6 K bytes)
2024-12-09 18:30:45,613 - stpipe - INFO - PARS-TWEAKREGSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-tweakregstep_0036.asdf
2024-12-09 18:30:45,625 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 18:30:45,634 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-sourcecatalogstep_0007.asdf 992 bytes (1 / 1 files) (0 / 992 bytes)
2024-12-09 18:30:45,863 - stpipe - INFO - PARS-SOURCECATALOGSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-sourcecatalogstep_0007.asdf
2024-12-09 18:30:45,878 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created.
2024-12-09 18:30:45,879 - stpipe.Image3Pipeline.assign_mtwcs - INFO - AssignMTWcsStep instance created.
2024-12-09 18:30:45,881 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created.
2024-12-09 18:30:45,882 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created.
2024-12-09 18:30:45,884 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created.
2024-12-09 18:30:45,885 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 18:30:45,887 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created.
2024-12-09 18:30:46,074 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('./nrc_im_demo_data/Obs001/jw02739-a3001_image3_00008_asn.json',).
2024-12-09 18:30:46,088 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_im_demo_data/Obs001/stage3
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
steps:
assign_mtwcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: False
suffix: assign_mtwcs
search_output_file: True
input_dir: ''
tweakreg:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_catalogs: False
use_custom_catalogs: False
catalog_format: ecsv
catfile: ''
starfinder: iraf
snr_threshold: 10.0
bkg_boxsize: 400
kernel_fwhm: 2.141
minsep_fwhm: 0.0
sigma_radius: 1.5
sharplo: 0.2
sharphi: 1.0
roundlo: -1.0
roundhi: 1.0
brightest: 200
peakmax: None
npixels: 10
connectivity: '8'
nlevels: 32
contrast: 0.001
multithresh_mode: exponential
localbkg_width: 0
apermask_method: correct
kron_params: None
enforce_user_order: False
expand_refcat: False
minobj: 15
fitgeometry: shift
nclip: 3
sigma: 3.0
searchrad: 2.0
use2dhist: True
separation: 2.0
tolerance: 1.0
xoffset: 0.0
yoffset: 0.0
abs_refcat: ''
save_abs_catalog: False
abs_minobj: 15
abs_fitgeometry: rshift
abs_nclip: 3
abs_sigma: 3.0
abs_searchrad: 6.0
abs_use2dhist: True
abs_separation: 1.0
abs_tolerance: 0.7
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
skymatch:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
skymethod: match
match_down: True
subtract: False
stepsize: None
skystat: mode
dqbits: ~DO_NOT_USE+NON_SCIENCE
lower: None
upper: None
nclip: 5
lsigma: 4.0
usigma: 4.0
binwidth: 0.1
outlier_detection:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: False
input_dir: ''
weight_type: ivm
pixfrac: 1.0
kernel: square
fillval: INDEF
nlow: 0
nhigh: 0
maskpt: 0.7
snr: 5.0 4.0
scale: 1.2 0.7
backg: 0.0
kernel_size: 7 7
threshold_percent: 99.8
rolling_window_width: 25
ifu_second_check: False
save_intermediate_results: False
resample_data: True
good_bits: ~DO_NOT_USE
allowed_memory: None
in_memory: False
resample:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: exptime
output_shape: None
crpix: None
crval: None
rotation: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
allowed_memory: None
in_memory: True
source_catalog:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: cat
search_output_file: True
input_dir: ''
bkg_boxsize: 1000
kernel_fwhm: 2.141
snr_threshold: 3.0
npixels: 25
deblend: False
aperture_ee1: 30
aperture_ee2: 50
aperture_ee3: 70
ci1_star_threshold: 2.0
ci2_star_threshold: 1.8
2024-12-09 18:30:46,100 - stpipe.Image3Pipeline - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename. All input files should be in a single directory, so no path is needed.
2024-12-09 18:30:46,100 - stpipe.Image3Pipeline - WARNING - warnings.warn(err_str, UserWarning)
2024-12-09 18:30:46,100 - stpipe.Image3Pipeline - WARNING -
2024-12-09 18:30:46,232 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca2_cal.fits' reftypes = ['abvegaoffset', 'apcorr']
2024-12-09 18:30:46,235 - stpipe.Image3Pipeline - INFO - Prefetch for ABVEGAOFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf'.
2024-12-09 18:30:46,236 - stpipe.Image3Pipeline - INFO - Prefetch for APCORR reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits'.
2024-12-09 18:30:46,237 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ...
2024-12-09 18:30:46,249 - stpipe.Image3Pipeline - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename. All input files should be in a single directory, so no path is needed.
2024-12-09 18:30:46,249 - stpipe.Image3Pipeline - WARNING - warnings.warn(err_str, UserWarning)
2024-12-09 18:30:46,249 - stpipe.Image3Pipeline - WARNING -
2024-12-09 18:30:46,936 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg running with args (<ModelContainer>,).
2024-12-09 18:30:48,877 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00001_nrca2_cal.fits.
2024-12-09 18:30:50,581 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00001_nrca4_cal.fits.
2024-12-09 18:30:52,441 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00002_nrca2_cal.fits.
2024-12-09 18:30:54,157 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00002_nrca4_cal.fits.
2024-12-09 18:30:54,181 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:30:54,182 - stpipe.Image3Pipeline.tweakreg - INFO - Number of image groups to be aligned: 2.
2024-12-09 18:30:54,182 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:30:54,182 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() started on 2024-12-09 18:30:54.182391
2024-12-09 18:30:54,183 - stpipe.Image3Pipeline.tweakreg - INFO - Version 0.8.9
2024-12-09 18:30:54,183 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:30:55,086 - stpipe.Image3Pipeline.tweakreg - INFO - Selected image 'GROUP ID: jw02739001002_02105_1' as reference image
2024-12-09 18:30:55,092 - stpipe.Image3Pipeline.tweakreg - INFO - Aligning image catalog 'GROUP ID: jw02739001002_02105_2' to the reference catalog.
2024-12-09 18:30:55,135 - stpipe.Image3Pipeline.tweakreg - INFO - Matching sources from 'jw02739001002_02105_00002_nrca' catalog with sources from the reference 'jw02739001002_02105_00001_nrca' catalog.
2024-12-09 18:30:55,135 - stpipe.Image3Pipeline.tweakreg - INFO - Computing initial guess for X and Y shifts...
2024-12-09 18:30:55,137 - stpipe.Image3Pipeline.tweakreg - INFO - Found initial X and Y shifts of 0.0007509, 0.000648 (arcsec) with significance of 288.9 and 299 matches.
2024-12-09 18:30:55,138 - stpipe.Image3Pipeline.tweakreg - INFO - Found 220 matches for 'GROUP ID: jw02739001002_02105_2'...
2024-12-09 18:30:55,139 - stpipe.Image3Pipeline.tweakreg - INFO - Performing 'shift' fit
2024-12-09 18:30:55,141 - stpipe.Image3Pipeline.tweakreg - INFO - Computed 'shift' fit for GROUP ID: jw02739001002_02105_2:
2024-12-09 18:30:55,141 - stpipe.Image3Pipeline.tweakreg - INFO - XSH: -2.40939e-05 YSH: -0.00118595
2024-12-09 18:30:55,142 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:30:55,142 - stpipe.Image3Pipeline.tweakreg - INFO - FIT RMSE: 0.00335582 FIT MAE: 0.00300579
2024-12-09 18:30:55,142 - stpipe.Image3Pipeline.tweakreg - INFO - Final solution based on 214 objects.
2024-12-09 18:30:55,207 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:30:55,207 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() ended on 2024-12-09 18:30:55.207203
2024-12-09 18:30:55,207 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() TOTAL RUN TIME: 0:00:01.024812
2024-12-09 18:30:55,208 - stpipe.Image3Pipeline.tweakreg - INFO -
2024-12-09 18:30:55,277 - stpipe.Image3Pipeline.tweakreg - INFO - Update S_REGION to POLYGON ICRS 274.714148067 -13.874666865 274.732194258 -13.875238842 274.732872320 -13.857853526 274.714793067 -13.857090476
2024-12-09 18:30:55,279 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:30:55,280 - stpipe.Image3Pipeline.tweakreg - WARNING - Use FITS instead.
2024-12-09 18:30:55,280 - stpipe.Image3Pipeline.tweakreg - WARNING - warnings.warn(
2024-12-09 18:30:55,280 - stpipe.Image3Pipeline.tweakreg - WARNING -
2024-12-09 18:30:55,281 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:30:55,281 - stpipe.Image3Pipeline.tweakreg - WARNING - Use to_string() instead.
2024-12-09 18:30:55,282 - stpipe.Image3Pipeline.tweakreg - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:30:55,282 - stpipe.Image3Pipeline.tweakreg - WARNING -
2024-12-09 18:30:55,385 - stpipe.Image3Pipeline.tweakreg - INFO - Update S_REGION to POLYGON ICRS 274.714842292 -13.855929606 274.732930692 -13.856595359 274.733745223 -13.839137146 274.715569489 -13.838277336
2024-12-09 18:30:55,388 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 18:30:55,388 - stpipe.Image3Pipeline.tweakreg - WARNING - Use FITS instead.
2024-12-09 18:30:55,388 - stpipe.Image3Pipeline.tweakreg - WARNING - warnings.warn(
2024-12-09 18:30:55,389 - stpipe.Image3Pipeline.tweakreg - WARNING -
2024-12-09 18:30:55,389 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 18:30:55,389 - stpipe.Image3Pipeline.tweakreg - WARNING - Use to_string() instead.
2024-12-09 18:30:55,390 - stpipe.Image3Pipeline.tweakreg - WARNING - cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 18:30:55,390 - stpipe.Image3Pipeline.tweakreg - WARNING -
2024-12-09 18:30:55,429 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2906: UserWarning: Double sampling check FAILED: Sampling may be too coarse for the distortion model being fitted.
2024-12-09 18:30:55,430 - stpipe.Image3Pipeline.tweakreg - WARNING - warnings.warn(
2024-12-09 18:30:55,430 - stpipe.Image3Pipeline.tweakreg - WARNING -
2024-12-09 18:30:55,452 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg done
2024-12-09 18:30:55,657 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch running with args (<ModelContainer>,).
2024-12-09 18:30:55,771 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:30:55,772 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() started on 2024-12-09 18:30:55.771590
2024-12-09 18:30:55,772 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:30:55,772 - stpipe.Image3Pipeline.skymatch - INFO - Sky computation method: 'match'
2024-12-09 18:30:55,773 - stpipe.Image3Pipeline.skymatch - INFO - Sky matching direction: DOWN
2024-12-09 18:30:55,773 - stpipe.Image3Pipeline.skymatch - INFO - Sky subtraction from image data: OFF
2024-12-09 18:30:55,773 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:30:55,774 - stpipe.Image3Pipeline.skymatch - INFO - ---- Computing differences in sky values in overlapping regions.
2024-12-09 18:30:57,611 - stpipe.Image3Pipeline.skymatch - INFO - * Group ID=1. Sky background of component images:
2024-12-09 18:30:57,612 - stpipe.Image3Pipeline.skymatch - INFO - - Image ID=jw02739001002_02105_00001_nrca2_cal.fits. Sky background: 0.0258186
2024-12-09 18:30:57,612 - stpipe.Image3Pipeline.skymatch - INFO - - Image ID=jw02739001002_02105_00001_nrca4_cal.fits. Sky background: 0.0258186
2024-12-09 18:30:57,612 - stpipe.Image3Pipeline.skymatch - INFO - * Group ID=2. Sky background of component images:
2024-12-09 18:30:57,613 - stpipe.Image3Pipeline.skymatch - INFO - - Image ID=jw02739001002_02105_00002_nrca2_cal.fits. Sky background: 0
2024-12-09 18:30:57,614 - stpipe.Image3Pipeline.skymatch - INFO - - Image ID=jw02739001002_02105_00002_nrca4_cal.fits. Sky background: 0
2024-12-09 18:30:57,614 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:30:57,614 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() ended on 2024-12-09 18:30:57.614352
2024-12-09 18:30:57,615 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() TOTAL RUN TIME: 0:00:01.842762
2024-12-09 18:30:57,615 - stpipe.Image3Pipeline.skymatch - INFO -
2024-12-09 18:30:57,624 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch done
2024-12-09 18:30:57,813 - stpipe.Image3Pipeline.outlier_detection - INFO - Step outlier_detection running with args (<ModelContainer>,).
2024-12-09 18:30:57,820 - stpipe.Image3Pipeline.outlier_detection - INFO - Performing outlier detection on 4 inputs
2024-12-09 18:30:57,820 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter kernel: square
2024-12-09 18:30:57,821 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:30:57,821 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter fillval: INDEF
2024-12-09 18:30:57,821 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter weight_type: ivm
2024-12-09 18:30:57,822 - stpipe.Image3Pipeline.outlier_detection - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:30:57,873 - stpipe.Image3Pipeline.outlier_detection - INFO - Computed output pixel scale: 0.030776005311869827 arcsec.
2024-12-09 18:30:57,947 - stpipe.Image3Pipeline.outlier_detection - INFO - 2 exposures to drizzle together
2024-12-09 18:31:00,050 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:02,946 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:03,989 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_outlier_i2d.fits
2024-12-09 18:31:04,001 - stpipe.Image3Pipeline.outlier_detection - INFO - 2 exposures to drizzle together
2024-12-09 18:31:06,661 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:10,131 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:11,176 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_outlier_i2d.fits
2024-12-09 18:31:11,316 - stpipe.Image3Pipeline.outlier_detection - INFO - Computing median
2024-12-09 18:31:23,175 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_outlier_i2d.fits
2024-12-09 18:31:23,201 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_outlier_i2d.fits
2024-12-09 18:31:23,205 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting median
2024-12-09 18:31:25,348 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2422, 4267)
2024-12-09 18:31:25,679 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_blot.fits
2024-12-09 18:31:27,793 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2422, 4267)
2024-12-09 18:31:28,133 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca4_a3001_blot.fits
2024-12-09 18:31:30,838 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2422, 4267)
2024-12-09 18:31:31,195 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_blot.fits
2024-12-09 18:31:33,881 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2422, 4267)
2024-12-09 18:31:34,448 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca4_a3001_blot.fits
2024-12-09 18:31:34,451 - stpipe.Image3Pipeline.outlier_detection - INFO - Flagging outliers
2024-12-09 18:31:34,829 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 161 (0.00%)
2024-12-09 18:31:35,167 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 49 (0.00%)
2024-12-09 18:31:35,524 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 133 (0.00%)
2024-12-09 18:31:35,880 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 65 (0.00%)
2024-12-09 18:31:35,907 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_blot.fits
2024-12-09 18:31:35,931 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca4_a3001_blot.fits
2024-12-09 18:31:35,956 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_blot.fits
2024-12-09 18:31:35,981 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca4_a3001_blot.fits
2024-12-09 18:31:36,244 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_crf.fits
2024-12-09 18:31:36,501 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca4_a3001_crf.fits
2024-12-09 18:31:36,786 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_crf.fits
2024-12-09 18:31:37,070 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca4_a3001_crf.fits
2024-12-09 18:31:37,071 - stpipe.Image3Pipeline.outlier_detection - INFO - Step outlier_detection done
2024-12-09 18:31:37,267 - stpipe.Image3Pipeline.resample - INFO - Step resample running with args (<ModelContainer>,).
2024-12-09 18:31:37,274 - stpipe.Image3Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 18:31:37,274 - stpipe.Image3Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 18:31:37,275 - stpipe.Image3Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 18:31:37,275 - stpipe.Image3Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 18:31:37,275 - stpipe.Image3Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 18:31:37,327 - stpipe.Image3Pipeline.resample - INFO - Computed output pixel scale: 0.030776005311869827 arcsec.
2024-12-09 18:31:37,382 - stpipe.Image3Pipeline.resample - INFO - Blending metadata for None
2024-12-09 18:31:37,927 - stpipe.Image3Pipeline.resample - INFO - Resampling science data
2024-12-09 18:31:39,970 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:42,823 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:46,250 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:49,662 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:50,495 - stpipe.Image3Pipeline.resample - INFO - Resampling variance components
2024-12-09 18:31:52,562 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:55,524 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:31:58,476 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:01,417 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:04,386 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:07,342 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:10,874 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:14,407 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:17,916 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:21,421 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:24,964 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:28,486 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 18:32:29,529 - stpipe.Image3Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 274.711011211 -13.874555187 274.732323507 -13.875342618 274.733750577 -13.838890827 274.712441621 -13.838103520
2024-12-09 18:32:30,230 - stpipe.Image3Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/image3_sw_i2d.fits
2024-12-09 18:32:30,231 - stpipe.Image3Pipeline.resample - INFO - Step resample done
2024-12-09 18:32:30,421 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog running with args (<ImageModel(2422, 4267) from image3_sw_i2d.fits>,).
2024-12-09 18:32:30,437 - stpipe.Image3Pipeline.source_catalog - INFO - Using APCORR reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits
2024-12-09 18:32:30,445 - stpipe.Image3Pipeline.source_catalog - INFO - Using ABVEGAOFFSET reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf
2024-12-09 18:32:30,445 - stpipe.Image3Pipeline.source_catalog - INFO - Instrument: NIRCAM
2024-12-09 18:32:30,446 - stpipe.Image3Pipeline.source_catalog - INFO - Detector: MULTIPLE
2024-12-09 18:32:30,446 - stpipe.Image3Pipeline.source_catalog - INFO - Filter: F200W
2024-12-09 18:32:30,446 - stpipe.Image3Pipeline.source_catalog - INFO - Pupil: CLEAR
2024-12-09 18:32:30,447 - stpipe.Image3Pipeline.source_catalog - INFO - Subarray: FULL
2024-12-09 18:32:30,468 - stpipe.Image3Pipeline.source_catalog - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 18:32:30,468 - stpipe.Image3Pipeline.source_catalog - WARNING - warnings.warn(
2024-12-09 18:32:30,469 - stpipe.Image3Pipeline.source_catalog - WARNING -
2024-12-09 18:32:30,486 - stpipe.Image3Pipeline.source_catalog - INFO - AB to Vega magnitude offset 1.68628
2024-12-09 18:32:36,771 - stpipe.Image3Pipeline.source_catalog - INFO - Detected 4336 sources
2024-12-09 18:32:40,453 - stpipe.Image3Pipeline.source_catalog - INFO - Wrote source catalog: ./nrc_im_demo_data/Obs001/stage3/image3_sw_cat.ecsv
2024-12-09 18:32:40,632 - stpipe.Image3Pipeline.source_catalog - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/image3_sw_segm.fits
2024-12-09 18:32:40,633 - stpipe.Image3Pipeline.source_catalog - INFO - Wrote segmentation map: image3_sw_segm.fits
2024-12-09 18:32:40,637 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog done
2024-12-09 18:32:40,639 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline done
# Print out the time benchmark
time1 = time.perf_counter()
print(f"Runtime so far: {time1 - time0:0.0f} seconds")
print(f"Runtime for Image3: {time1 - time_image3:0.0f} seconds")
Runtime so far: 1156 seconds
Runtime for Image3: 173 seconds
Verify which pipeline steps were run#
# Identify *_i2d file and open as datamodel
if doimage3:
if do_swimage3:
sw_i2d_file = os.path.join(image3_dir, f'{sw_product_name}_i2d.fits')
i2d_sw_model = datamodels.open(sw_i2d_file)
step_check_model = i2d_sw_model
if do_lwimage3:
lw_i2d_file = os.path.join(image3_dir, f'{lw_product_name}_i2d.fits')
i2d_lw_model = datamodels.open(lw_i2d_file)
step_check_model = i2d_lw_model
# Check which steps were run. This should be the same regardless of whether
# a sw or lw file is used.
step_check_model.meta.cal_step.instance
{'assign_wcs': 'COMPLETE',
'charge_migration': 'SKIPPED',
'dark_sub': 'COMPLETE',
'dq_init': 'COMPLETE',
'flat_field': 'COMPLETE',
'gain_scale': 'SKIPPED',
'group_scale': 'SKIPPED',
'ipc': 'SKIPPED',
'jump': 'COMPLETE',
'linearity': 'COMPLETE',
'outlier_detection': 'COMPLETE',
'persistence': 'SKIPPED',
'photom': 'COMPLETE',
'ramp_fit': 'COMPLETE',
'refpix': 'COMPLETE',
'resample': 'COMPLETE',
'saturation': 'COMPLETE',
'skymatch': 'COMPLETE',
'superbias': 'COMPLETE',
'tweakreg': 'COMPLETE'}
Check which reference files were used to calibrate the dataset
if doimage3:
step_check_model.meta.ref_file.instance
{'area': {'name': 'crds://jwst_nircam_area_0261.fits'},
'camera': {'name': 'N/A'},
'collimator': {'name': 'N/A'},
'crds': {'context_used': 'jwst_1293.pmap', 'sw_version': '12.0.8'},
'dark': {'name': 'crds://jwst_nircam_dark_0424.fits'},
'dflat': {'name': 'N/A'},
'disperser': {'name': 'N/A'},
'distortion': {'name': 'crds://jwst_nircam_distortion_0277.asdf'},
'fflat': {'name': 'N/A'},
'filteroffset': {'name': 'crds://jwst_nircam_filteroffset_0007.asdf'},
'flat': {'name': 'crds://jwst_nircam_flat_0634.fits'},
'fore': {'name': 'N/A'},
'fpa': {'name': 'N/A'},
'gain': {'name': 'crds://jwst_nircam_gain_0097.fits'},
'ifufore': {'name': 'N/A'},
'ifupost': {'name': 'N/A'},
'ifuslicer': {'name': 'N/A'},
'linearity': {'name': 'crds://jwst_nircam_linearity_0052.fits'},
'mask': {'name': 'crds://jwst_nircam_mask_0076.fits'},
'msa': {'name': 'N/A'},
'ote': {'name': 'N/A'},
'photom': {'name': 'crds://jwst_nircam_photom_0157.fits'},
'readnoise': {'name': 'crds://jwst_nircam_readnoise_0266.fits'},
'regions': {'name': 'N/A'},
'saturation': {'name': 'crds://jwst_nircam_saturation_0097.fits'},
'sflat': {'name': 'N/A'},
'specwcs': {'name': 'N/A'},
'superbias': {'name': 'crds://jwst_nircam_superbias_0220.fits'},
'wavelengthrange': {'name': 'N/A'}}
8. Visualize the resampled images#
If you specified that the LW and SW outputs should be resampled onto the same pixel grid, you should be able to open the two i2d files and overlay them and see that the sources and pixel grids line up. If there is any misalignment, you may need to adjust tweakreg parameters in the calls to the Image3 pipeline in order to improve the alignment.
Below we use the Imviz tool within the jdaviz package to visualize the images, one filter at a time.
# Create an Imviz instance and set up default viewer for the F200W data
if doimage3 and do_swimage3:
imviz_sw_i2d = Imviz()
viewer_sw_i2d = imviz_sw_i2d.default_viewer
# Read in the science array for our visualization dataset:
i2d_sw_science = i2d_sw_model.data
# Load the dataset into Imviz
imviz_sw_i2d.load_data(i2d_sw_science)
# Visualize the dataset:
imviz_sw_i2d.show()
Remember that in this mosaic we have only two detectors: NRC2 and NRC4 (left and right, respectively). The dither is not large enough to cover the gap between the detectors, and so that gap is still visible in the mosaic.
if doimage3 and do_swimage3:
viewer_sw_i2d.stretch = 'sqrt'
viewer_sw_i2d.set_colormap('Viridis')
viewer_sw_i2d.cuts = '95%'
# Create an Imviz instance and set up default viewer for the F444W data
if doimage3 and do_lwimage3:
imviz_lw_i2d = Imviz()
viewer_lw_i2d = imviz_lw_i2d.default_viewer
# Read in the science array for our visualization dataset:
i2d_lw_science = i2d_lw_model.data
# Load the dataset into Imviz
imviz_lw_i2d.load_data(i2d_lw_science)
# Visualize the dataset:
imviz_lw_i2d.show()
if doimage3 and do_lwimage3:
viewer_lw_i2d.stretch = 'sqrt'
viewer_lw_i2d.set_colormap('Viridis')
viewer_lw_i2d.cuts = '95%'
Ovelaying the LW and SW images#
Let’s try putting the SW and LW images on top of one another to create a color image. This should work regardless of whether you resampled the two images onto the same pixel grid.
Let’s get the data first
if doimage3 and do_swimage3 and do_lwimage3:
imviz_color = Imviz()
viewer_color = imviz_color.default_viewer
# Load the datasets into Imviz
imviz_color.load_data(sw_i2d_file, data_label='sw')
imviz_color.load_data(lw_i2d_file, data_label='lw')
# Link images by WCS (without affine approximation)
imviz_color.plugins['Links Control'].link_type = 'WCS'
imviz_color.plugins['Links Control'].wcs_use_affine = False
2024-12-09 18:32:44,635 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jdaviz/core/user_api.py:49: AstropyDeprecationWarning: The link_type function is deprecated and may be removed in a future version.
2024-12-09 18:32:44,635 - stpipe - WARNING - Use align_by instead.
2024-12-09 18:32:44,636 - stpipe - WARNING - exp_obj = getattr(self._obj, attr)
2024-12-09 18:32:44,636 - stpipe - WARNING -
2024-12-09 18:32:45,325 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jdaviz/core/user_api.py:49: AstropyDeprecationWarning: The wcs_use_affine function is deprecated and may be removed in a future version.
2024-12-09 18:32:45,325 - stpipe - WARNING - Use wcs_fast_approximation instead.
2024-12-09 18:32:45,325 - stpipe - WARNING - exp_obj = getattr(self._obj, attr)
2024-12-09 18:32:45,326 - stpipe - WARNING -
2024-12-09 18:32:45,326 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jdaviz/core/user_api.py:85: AstropyDeprecationWarning: The wcs_use_affine function is deprecated and may be removed in a future version.
2024-12-09 18:32:45,326 - stpipe - WARNING - Use wcs_fast_approximation instead.
2024-12-09 18:32:45,327 - stpipe - WARNING - return setattr(self._obj, attr, value)
2024-12-09 18:32:45,327 - stpipe - WARNING -
Now define some options to make the picture look nice.
# Set the colors for the two images.
if doimage3 and do_swimage3 and do_lwimage3:
plot_options = imviz_color.plugins['Plot Options']
plot_options.image_color_mode = 'Color'
img_settings = {'sw': {'image_color': '#61d3e1',
#'stretch_vmin': 0,
#'stretch_vmax': 4,
#'image_opacity': 0.32,
#'image_contrast': 0.69,
#'image_bias': 0.39
},
'lw': {'image_color': '#ff767c',
#'stretch_vmin': 0,
#'stretch_vmax': 16,
#'image_opacity': 0.4,
#'image_contrast': 0.94,
#'image_bias': 0.74
}
}
Populate the imviz instance with the settings in the cell above and visualize the dataset
# Now populate the imviz instance with the settings in the cell above.
if doimage3 and do_swimage3 and do_lwimage3:
for layer, settings in img_settings.items():
plot_options.layer = f'{layer}[DATA]'
for k, v in settings.items():
setattr(plot_options, k, v)
# Visualize the dataset
if doimage3 and do_swimage3 and do_lwimage3:
imviz_color.show()
9. Visualize Detected Sources#
Using the source catalogs created by the Image3 stage of the pipeline, mark the detected sources, using different markers for point sources and extended sources. The source catalogs are saved in image3/image3_sw_cat.ecsv and image3/image3_lw_cat.ecsv. This time, we will provide the i2d filename to the imviz load_data function, rather than just the array of pixel values. This way, imviz will be able to make use of the WCS in the file. This will allow the sources in the source catalog to be accurately marked in the display.
Read in catalog file and identify point/extended sources#
if doimage3:
if do_swimage3:
sw_catalog_file = sw_i2d_file.replace('i2d.fits', 'cat.ecsv')
sw_catalog = Table.read(sw_catalog_file)
# To identify point/extended sources, use the 'is_extended' column in the source catalog
sw_pt_src, = np.where(~sw_catalog['is_extended'])
sw_ext_src, = np.where(sw_catalog['is_extended'])
# Define coordinates of point and extended sources
sw_pt_coord = Table({'coord': [SkyCoord(ra=sw_catalog['sky_centroid'][sw_pt_src].ra,
dec=sw_catalog['sky_centroid'][sw_pt_src].dec)]})
sw_ext_coord = Table({'coord': [SkyCoord(ra=sw_catalog['sky_centroid'][sw_ext_src].ra,
dec=sw_catalog['sky_centroid'][sw_ext_src].dec)]})
if do_lwimage3:
lw_catalog_file = lw_i2d_file.replace('i2d.fits', 'cat.ecsv')
lw_catalog = Table.read(lw_catalog_file)
# To identify point/extended sources, use the 'is_extended' column in the source catalog
lw_pt_src, = np.where(~lw_catalog['is_extended'])
lw_ext_src, = np.where(lw_catalog['is_extended'])
# Define coordinates of point and extended sources
lw_pt_coord = Table({'coord': [SkyCoord(ra=lw_catalog['sky_centroid'][lw_pt_src].ra,
dec=lw_catalog['sky_centroid'][lw_pt_src].dec)]})
lw_ext_coord = Table({'coord': [SkyCoord(ra=lw_catalog['sky_centroid'][lw_ext_src].ra,
dec=lw_catalog['sky_centroid'][lw_ext_src].dec)]})
Mark the extended and point sources on the images#
Display the image with sources indicated by circles. Point sources will be marked by small pink circles and extended sources will be marked by white circles. Looking at the entire mosaic, there are so many sources found that it’s hard to see much of anything. To get a clearer view, try zooming in on various areas using the magnifying glass icon on the banner immediately above the image.
First we visualize the data without the point sources.
# Read in SW i2d file to Imviz
if doimage3 and do_swimage3:
imviz_sw_cat = Imviz()
viewer_sw_cat = imviz_sw_cat.default_viewer
imviz_sw_cat.load_data(sw_i2d_file)
# Adjust settings for viewer
viewer_sw_cat.stretch = 'sqrt'
viewer_sw_cat.set_colormap('Viridis')
viewer_sw_cat.cuts = '95%'
imviz_sw_cat.show()
Now we add the point sources
# Add marker for point sources:
if doimage3 and do_swimage3:
viewer_sw_cat.marker = {'color': 'pink', 'markersize': 50, 'fill': False}
viewer_sw_cat.add_markers(sw_pt_coord, use_skycoord=True, marker_name='point_sources')
# Add marker for extended sources:
viewer_sw_cat.marker = {'color': 'white', 'markersize': 100, 'fill': False}
viewer_sw_cat.add_markers(sw_ext_coord, use_skycoord=True, marker_name='extended_sources')
We do the same with the LW file. First we visualize the data.
# Repeat using the LW file
if doimage3 and do_lwimage3:
imviz_lw_cat = Imviz()
viewer_lw_cat = imviz_lw_cat.default_viewer
imviz_lw_cat.load_data(lw_i2d_file)
# Adjust settings for viewer
viewer_lw_cat.stretch = 'sqrt'
viewer_lw_cat.set_colormap('Viridis')
viewer_lw_cat.cuts = '95%'
imviz_lw_cat.show()
Now we mark the point sources
# Add marker for point sources:
if doimage3 and do_lwimage3:
viewer_lw_cat.marker = {'color': 'pink', 'markersize': 50, 'fill': False}
viewer_lw_cat.add_markers(lw_pt_coord, use_skycoord=True, marker_name='point_sources')
# Add marker for extended sources:
viewer_lw_cat.marker = {'color': 'white', 'markersize': 100, 'fill': False}
viewer_lw_cat.add_markers(lw_ext_coord, use_skycoord=True, marker_name='extended_sources')
10. Notes#
Note that the strategy presented in this notebook for placing the SW data onto the same pixel grid as the LW data can be applied to data from any two datasets, regardless of filter or channel. By saving the gWCS from the first dataset into an asdf file and providing that file to the
Image3call with the second dataset, the resulting i2d images will be aligned onto the same pixel grid.If you notice poor alignment across tiles within a single i2d image, or between i2d images that you expect to be aligned, try adjusting the parameters in the
tweakregstep. With these, you can customize which sourcestweakregidentifies and uses for the alignment.
